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Executive  Summary 


Introduction 


This  report  presents  the  results  of  an  analysis  of  the  federal  government’s  investment  in  learning 
and  education  research  within  the  domains  of  science,  technology,  engineering  and  mathematics 
(STEM).  The  effort  to  assess  the  federal  STEM  education  research  portfolio  originated  as  part  of 
a  more  expansive  set  of  activities  under  the  direction  of  The  National  Science  and  Technology 
Council’s  (NSTC)  Education  and  Workforce  Development  Subcommittee  (EWD).  The  broad 
purpose  and  scope  of  the  EWD  Subcommittee  was:  “to  define  the  current  and  future  need  to 
attract,  retain  and  retrain  workers  that  provide  the  critical  Science,  Technology,  Engineering  and 
Mathematics  (STEM)  capabilities  required  within  the  Federal  workforce,  to  identify  and  support 
Federal  efforts  to  contribute  to  the  national  STEM  workforce,  and  to  propose  solutions,  actions 
and  initiatives  that  address  those  needs.”  In  contrast,  the  more  focused  charges  to  the  Task  Group 
assigned  to  examine  STEM  education  research  were  to: 

•  Review  and  appraise  the  depth  and  content  of  the  current  federal  investment  in  research 
on  learning  and  education  in  science,  technology,  engineering  and  mathematics  (STEM) 
and  in  evaluation  research,  K-20. 

•  Provide  recommendations  for  strengthening  the  federal  education  research  portfolio. 

Among  other  objectives,  a  major  goal  of  this  report  is  to  stimulate  the  development  of  federal 
research  agendas  both  within  and  among  agencies  that  will  lead  to  the  strengthening  of  our 
knowledge  base  in  this  field  in  the  short  run,  and  to  improvements  in  STEM  learning  and 
educational  practices  in  the  long  run. 

Snapshot  of  the  Federal  STEM  Education  Research  Portfolio  —  FY  2003 

For  the  purposes  of  this  report,  research  was  defined  as  any  systematic  investigation  designed  to 
develop  or  contribute  to  the  knowledge  base  pertaining  to  STEM  education  and  learning. 
Excluded  from  the  analysis  were  demonstration  projects,  reviews  of  the  literature,  meta-analytic 
studies,  and  routine  evaluations  of  curriculum  projects  or  programs.  However,  evaluation 
research  projects  were  included  if  they  qualified  as  compelling  evaluative  studies  that  build  the 
knowledge  base  about  effective  STEM  education  policy  or  practice.  The  Task  Group  then 
identified  three  primary  agencies  that  have  a  significant  and  targeted  focus  on  funding  research 
in  STEM  education:  the  U.S.  Department  of  Education  (ED),  the  National  Institutes  of  Health 
(NIH),  and  the  National  Science  Foundation  (NSF).  It  was  decided  that  the  most  reasonable 
approach  to  describing  the  Federal  STEM  education  research  portfolio  was  to  present  a  snapshot 
of  the  relevant  programs  and  expenditures  of  these  three  agencies  for  one  fiscal  year.  FY  2003 
was  chosen  as  this  the  most  recent  year  for  which  the  requisite  financial  investment  data  were 
available  at  the  time  the  Task  Group  began  its  work.  Then,  proceeding  according  to  the 
definitions  and  inclusion  criteria  described  above,  the  Task  Group  members  each  culled  their 
respective  agency’s  portfolios  and  identified  a  total  of  29  funding  programs  that  were  operative 
in  FY  2003,  14  programs  within  the  U.S.  Department  of  Education’s  Institute  of  Education 
Sciences  (IES),  two  programs  within  the  National  Institute  of  Child  Health  and  Human 


3 


Development  (NICHD)  at  the  NIH,  and  13  programs  sponsored  by  the  National  Science 
Foundation  (NSF). 

Overall  findings.  The  total  estimated  federal  investment  in  STEM  education  and  learning 
research  for  FY  2003  was  approximately  $190  million.  Breaking  this  down  by  agency  indicates 
that  in  FY  2003,  82  percent  of  funding  for  STEM  education  research  came  from  the  National 
Science  Foundation,  15  percent  from  the  U.S.  Department  of  Education,  and  3  percent  from  the 
National  Institute  of  Child  Health  and  Human  Development,  NIH.  Additionally,  35  percent  of 
the  $190  million  was  directed  toward  research  in  math  education,  25  percent  toward  science,  23 
percent  toward  technology,  7  percent  toward  engineering,  and  10  percent  toward  combined 
STEM  areas. 

Federal  Agency  Program  Objectives  for  STEM  Education  Research 

Next,  the  Task  Group  developed  a  framework  for  examining  the  objectives  that  have  been 
guiding  the  STEM  education  research  investments  of  the  three  major  federal  agencies.  This  was 
accomplished  by  reviewing  published  solicitations,  program  announcements,  and  program 
descriptions  to  detennine  the  intent  of  the  prospective  investments  of  each  of  the  relevant  agency 
programs  up  through  FY  2005. 

Foci  of  federal  program  objectives.  The  analysis  revealed  that  most  of  the  federal  STEM 
education  research  objectives  focus  on  curriculum  or  instructional  practice,  student  cognition  and 
learning,  and  evaluation  and  assessment.  Slightly  less  than  half  of  the  programs  include  various 
equity  issues  in  their  research  objectives,  and  for  the  most  part,  these  programs  appear  to  be 
specifically  focused  on  equity  issues.  Higher  education,  Faculty  development  and  STEM 
workplace  improvement  are  covered  by  a  number  of  programs,  all  which  are  in  the  NSF 
portfolio. 

Research  methods — agency  priorities.  During  the  past  several  years,  issues  concerning  the 
appropriateness  of  research  designs  and  methods  have  become  especially  prominent  in  the 
educational  research  community  as  well  as  in  educational  policy  circles.  Consequently,  the 
present  report  characterizes  the  current  objectives  and  preferences  of  NSF,  IES,  and  NICHD 
regarding  the  use  of  methods,  designs,  and  analysis  in  STEM  learning  and  educational  research. 
Even  a  cursory  reading  of  these  descriptions  reveals  one  common  principle  across  the  agencies  - 
that  the  research  method(s)  employed  should  fit  the  question(s)  at  hand,  which  is  also  consistent 
with  one  of  the  recommendations  put  forward  in  a  recently  published  National  Academy  report, 
Advancing  Scientific  Research  in  Education  (2005).  Beyond  this  guideline,  the  agencies  differ  to 
some  extent  in  their  preferences  for  various  approaches  to  STEM  educational  research.  At  the 
very  least,  it  is  equally  clear  that  a  wide  range  of  approaches  is  deemed  acceptable  by  one  or 
more  of  these  agencies. 

Conveying  STEM  Education  Research  Results  to  Policymakers  and  Practitioners 


Policymakers.  In  reviewing  how  the  three  major  federal  agencies  disseminate  research  results  to 
policymakers,  the  Task  Group  learned  that  for  the  most  part,  this  is  being  achieved  through  each 
agency’s  own  website,  the  projects’  websites,  an  agency  clearinghouse,  and  through  government 
reports.  Professional  conferences  and  scholarly  journals  are  the  next  most  commonly  used 
avenues  for  making  such  findings  available  to  the  public  and  policymakers.  Several  programs 
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strongly  encourage  the  recipients  of  their  grants  to  disseminate  findings  via  professional 
conferences  and  scholarly  publications.  In  general,  these  are  considered  passive  inodes  of 
delivery  because  policymakers,  educators  and  the  general  public  must  make  the  effort  to  find  and 
access  research  results.  In  some  instances,  federal  programs  deliver  findings  directly  to 
policymakers  via  policy  briefs,  testimony  to  congressional  subcommittees,  meetings  with  policy 
experts,  or  seminars. 

A  number  of  possible  action  steps  have  been  proposed  by  researchers  and  others  regarding  how 
federal  agencies  might  improve  the  integration  of  research  and  policy,  including  for  example:  a) 
support  periodic  research  syntheses  in  critical  areas  so  that  the  knowledge  base  can  be  made 
accessible  to  local  decision-makers.  Although  federal  agencies  have  funded  some  research 
syntheses,  for  example,  studies  conducted  by  the  National  Academy  of  Sciences,  the  redirecting 
of  investments  to  support  additional  efforts  of  this  type  may  well  prove  beneficial;  b)  provide 
research  in  forms  that  are  easily  digestible  by  non-researchers .  One  approach  for  making 
research  accessible  to  practitioners  and  policymakers  is  the  What  Works  Clearinghouse,  which 
was  established  in  2002  by  the  Institute  of  Education  Sciences.  The  What  Works  Clearinghouse 
provides  consumers  of  education  research  with  user-friendly  reviews  of  the  effectiveness  of 
replicable  education  interventions  (programs,  products,  practices,  and  policies)  that  are  intended 
to  improve  student  outcomes. 

Practitioners.  Drawing  on  the  work  of  various  scholars,  the  report  also  acknowledges  several 
barriers  that  prevent  teachers  from  making  use  of  educational  research  findings,  including  that 
most  research  evidence  is  published  in  places  and  forms  that  only  other  researchers  access  and 
comprehend;  much  of  this  research  lacks  specifically  applicability;  some  of  is  ambiguous;  and 
the  culture  of  teaching  does  not  typically  make  decisions  based  on  research  findings.  It  is 
suggested  that  instead  of  structuring  the  dissemination  activities  as  a  one-way  flow  of 
information,  more  sophisticated  dissemination  efforts  need  to  be  devised  and  examined  that 
could  involve  an  exchange  of  information  and  ideas  between  researchers  and  practitioners. 

Finally,  it  should  be  noted  that  efforts  by  federal  agencies  to  better  integrate  education  research 
with  practice  are  frequently  constrained  by  the  administrative  and  established  organizational 
structures  in  which  reforms  must  be  implemented.  In  this  regard,  more  research  is  needed  on  the 
factors  that  are  critical  for  successfully  bringing  about  large-scale  organizational  change  within 
educational  systems  so  that  future  efforts  to  effect  needed  reforms  will  be  founded  on  a  solid 
evidence  base. 

How  do  our  federal  educational  research  programs  currently  disseminate  research  findings  to 
practitioners?  Results  from  the  program  solicitations  reviewed  here  indicate  that  the  three  federal 
agencies  inform  practitioners  about  the  research  they  fund  primarily  through  conferences  and 
workshops,  journal  articles,  newsletters,  and  agency  websites.  While  individual  projects  may  be 
required  to  work  directly  and  closely  with  practitioners,  rarely  do  federal  programs  convey  the 
results  of  their  research  directly  to  practitioners. 

What  can  federal  educational  research  programs  do  to  better  integrate  research  with  practice? 

The  report  lists  a  number  of  ways  in  which  federal  agencies  can  help  to  better  integrate  education 
research  and  practice,  including: 
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•  Encourage  federally-funded  research  projects  to  communicate  their  findings  to  multiple 
audiences  (researchers,  practitioners,  policymakers). 

•  Engage  practitioners  to  collaborate  with  researchers  in  setting  research  agendas. 

•  Seek  well-designed  projects  that  engage  teachers  in  the  research  process. 

•  Engage  practitioners  along  with  researchers  as  peer  reviewers  in  every  milestone  of  the 
research  project’s  lifespan,  identifying  when  research-based  knowledge  is  good  enough 
to  inform  practice  and  policy. 

•  Support  projects  that  create  lasting  bonds  between  education  and  science  communities, 
moving  beyond  research  and  education  as  parallel  but  disconnected  activities. 

Analysis  of  Recommendations  Drawn  From  STEM  Education  Research  Reports 

The  Task  Group  reviewed  64  STEM  education  and  workforce  reports  issued  in  the  last  10  years 
in  an  effort  to  carefully  examine  the  kinds  concerns  raised  about  STEM  education  research.  The 
list  includes  numerous  reports  from  the  National  Academy  of  Sciences,  as  well  as  the  RAND 
Corporation,  the  Carnegie  Foundation,  the  Educational  Testing  Service,  the  National  Science 
Board,  and  the  White  House  Office  of  Science  and  Technology  Policy,  among  others.  Although 
literally  hundreds  of  STEM  education-related  issues,  ideas,  concerns,  suggestions,  and 
recommendations  were  extracted,  the  vast  majority  of  these  were  not  directed  toward  education 
research',  and  since  this  was  the  focus  of  the  Task  Group’s  charge,  only  recommendations 
pertaining  to  STEM  education  research  were  considered  in  preparing  the  present  report.  This  list 
of  recommendations  was  broken  down  into  two  main  categories — recommendations  about 
research  and  recommendations  about  processes  related  to  research. 

Research  recommendations.  With  respect  to  the  research  recommendations,  the  Task  Group 
examined  the  extent  to  which  the  program  objectives  of  the  three  major  federal  agencies  are 
addressing  these  satisfactorily,  and  where  gaps  remain  either  in  the  kind  of  work  currently  being 
supported  or  in  the  way  that  research  projects  are  solicited.  In  addition,  where  applicable,  the 
Task  Group  considered  the  efficiency  of  program  objectives  within  and  across  agencies,  focusing 
on  the  degree  of  overlap  and  whether  better  coordination  could  maximize  the  impact  of  joint 
investments.  The  research  categories  examined  include:  teachers,  student  learning,  assessment 
and  evaluation  of  interventions,  education  of  women  and  underrepresented  groups,  career 
patterns  of  undergraduate,  graduate  and  career  scientists,  and  communication  and  synthesis  of 
research  results. 

Teachers.  This  category  includes  research  pertaining  to  the  identification  of  effective  teacher 
licensure  tests;  dynamics  of  teacher  perfonnance  and  effectiveness;  practices  that  enable  teachers 
to  help  students  develop  mathematical  proficiency;  ways  to  develop  teachers’  knowledge  of 
mathematics  and  science;  and  new  ways  to  educate,  train  and  evaluate  teachers  in  the  use  and 
application  of  21st  century  technology  skills.  The  Task  Group  did  not  identify  any  major 
programmatic  gaps  in  this  area.  The  rather  extensive  focus  on  teacher  development  across  at 
least  half  of  the  agency  programs  examined  is  consistent  with  the  recommendations  coming  from 
many  quarters  regarding  an  urgent  need  to  improve  our  understanding  in  this  area.  However, 
greater  coordination  is  needed  to  ensure  that  these  efforts  are  complementary  rather  than 
duplicative.  A  more  detailed  analysis  is  needed  to  properly  assess  areas  that  would  benefit  from 
greater  coordination  among  agencies  and  investigators. 
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Student  learning.  This  cluster  includes  research  recommendations  about  identifying 
developmental  models  of  cognition  and  learning;  practices  that  are  essential  to  learning  and 
effective  day-to-day  use  of  mathematics;  large-scale  studies  to  investigate  the  impact  of 
standards-based  curricula  on  student  achievement;  and  consensus  on  a  common  core  of 
mathematics  and  science  knowledge  and  skills.  The  Task  Group  did  not  identify  any  major  gaps 
in  this  area.  Further,  the  Task  Group  determined  that  there  is  comparatively  little  overlap  in  the 
majority  of  the  kinds  of  research  being  supported  in  this  area  by  ED  and  NIH.  For  example,  the 
large  majority  of  NIH  studies  focus  on  laboratory-based,  cognitive  research  in  mathematical 
thinking  and  scientific  reasoning,  while  the  objectives  of  ED’s  IES  solicitations  tend  to  focus  on 
studies  in  the  context  of  everyday  instructional  settings,  building  upon  the  findings  of  the  NIH 
type  of  research.  There  appears  to  be  a  somewhat  greater,  albeit  limited  degree  of  overlap 
between  portions  of  the  NSF  ROLE  (Research  on  Learning  in  Education)  Program  and  the 
NIH/NICHD  Mathematics  and  Science  Cognition  and  Learning  portfolios.  These  programs 
should  be  carefully  coordinated  to  capitalize  on  opportunities  for  collaboration,  maximize  the 
impact  of  the  investments,  and  avoid  duplication  of  effort. 

Assessment  and  evaluation  of  interventions .  Included  in  this  category  is  research  focused  on  the 
design  of  assessments  that  yield  valid  and  fair  inferences  about  student  achievement  or 
instructional  interventions.  Despite  the  emphasis  on  assessment  in  the  programs  cited  in  the 
report,  it  is  clear  that  there  remain  significant  gaps  in  our  knowledge  about  assessment, 
especially  in  the  area  of  student  achievement.  As  with  the  other  areas  mentioned  above,  further 
analysis  of  opportunities  for  collaboration  between  programs  with  similar  objectives,  both  within 
and  across  agencies,  is  called  for. 

Education  of  women  and  underrepresented  groups.  Although  various  programs  at  ED,  NSF,  and 
NIH  target  different  issues  and  groups,  it  is  clear  that  more  could  be  done  in  some  of  these 
domains.  While  some  major  steps  are  being  made  to  reduce  racial  and  ethnicity  disparities,  to 
provide  new  opportunities  for  socio-economically  disadvantaged  students,  as  well  as  for 
increasing  participation  by  those  with  disabilities,  we  still  do  not  understand  how  best  to  reduce 
and  eliminate  the  barriers  various  underrepresented  groups  face  in  STEM  fields. 

Career  patterns  of  undergraduate,  graduate  and  career  scientists.  To  the  extent  that  their 
authority  permits,  the  agencies  should  consider  placing  higher  priority  on  this  area  than  is 
currently  the  case.  The  rapidly  growing  demand  for  workers  with  STEM  knowledge  highlights 
the  urgent  need  for  gaining  a  better  understanding  of  the  preparedness  and  career  choices  of 
young  people  entering  the  workforce.  We  suggest  that  a  federal  interagency  effort  may  be 
needed  to  efficiently  explore  linkages  between  STEM  workforce  research  and  education  research 
in  curriculum  and  instructional  practices,  equity,  and  student  cognition  and  learning.  A 
collaborative  approach  of  this  type  would  leverage  existing  resources  in  a  manner  that  could 
enhance  our  understanding  of  the  factors  that  give  rise  to  this  national  problem  and  provide 
evidence-based  corrective  actions  for  effectively  modifying  the  current  imbalance  between 
STEM  workforce  supply  and  demand. 

Communication  and  synthesis  of  research  results.  Despite  increased  efforts  in  this  area,  the 
members  of  the  Task  Group  agree  that  there  is  much  to  be  done  before  research  results  are 
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adequately  synthesized  in  a  manner  that  will  permit  effective  communication  to  both 
policymakers  and  practitioners,  among  other  audiences. 

Process  recommendations.  With  respect  to  process  recommendations,  the  task  group  found  that 
agencies  were  working  hard  to  improve  the  evaluation  of  programs  using  more  rigorous  methods 
and  designs,  including  a  wide  array  of  both  quantitative  and  qualitative  methods.  Similarly, 
agencies  have  enhanced  the  focus  and  selectivity  of  their  programs,  have  added  increasing 
emphasis  on  exploring  the  role  of  technology  in  education,  and  have  supported  the  development 
of  national  databases.  Process  recommendations  requiring  further  attention  include  developing  a 
STEM  education  federal  investment  database;  improving  communication  of  research  results; 
enhancing  syntheses  of  available  knowledge;  increasing  links  between  research,  policy,  and 
practice;  and  facilitating  greater  coordination  and  cooperation  among  federal  agencies. 

Recommendations  for  Federal  Interagency  Cooperation,  Coordination,  and  Collaboration 

Recommendations.  The  Task  Group  provided  a  list  of  specific  ways  in  which  the  federal 
agencies  can  work  together  to  increase  effectiveness  and  fill  gaps  in  the  current  STEM  education 
and  evaluation  research  portfolio.  At  a  minimum,  agencies  should:  keep  one  another  informed  of 
the  solicitations  being  planned  for  subsequent  fiscal  years;  incorporate  into  their  program 
solicitations  references  to  funding  opportunities  in  the  other  agencies;  maintain  a  common 
database  of  all  of  their  funded  STEM  education  research  projects,  perhaps  developing  and  using 
a  common  structured  abstract  format;  and  cooperate  by  discussing  the  domains  of  STEM 
education  research  where  each  agency  has  particular  strengths  and  then  collaborate  accordingly. 
The  Task  Group  also  recommends  that  agencies  collaborate  on  developing  and  publishing 
solicitations,  finding  new  ways  to  jointly  fund  and  evaluate  grants  issued  under  those 
solicitations,  and  co-sponsor  workshops  and  symposia  on  key  issues  related  to  STEM  education 
research  and  evaluation. 

Challenges.  As  straightforward  as  these  suggestions  may  appear,  they  do  pose  some  challenges. 
For  example,  the  three  agencies  have  different  “cultures”  and  different  kinds  of  connections  to 
and  relationships  with  the  field.  To  some  extent,  the  agencies  have  different  stakeholders,  and 
their  programs  cover  different  bands  on  the  education  continuum  (e.g.,  K-12,  community 
college,  undergraduate).  There  is  also  a  long  history  of  separate  institutional  goal  setting.  Despite 
these  differences,  cooperation  may  be  achieved  by  a  continuing  dialogue  among  these  agencies 
regarding  their:  a)  respective  strengths  and  priorities;  b)  strategic  long-term  plans;  and  c)  funding 
precedents  and  constraints  (legal,  institutional,  historical  or  otherwise).  Attempting  to  explore 
collaborative  and  cooperative  directions  would  at  the  very  least  require  continuing  interagency 
assessment  of  STEM  education  programs,  which  should  in  itself  enhance  the  efficiency  of 
federal  investments  in  these  areas.  However,  it  is  equally  evident  that  each  federal  agency  has  its 
own  mission.  Although  differences  in  this  regard  can  sometimes  hinder  collaborative  efforts, 
they  can  also  yield  a  potentially  richer  and  more  efficient  redirection  of  existing  resources  in  the 
service  of  common  objectives.  Therefore,  we  recommend  that  the  existing  NSTC  Education  and 
Workforce  Subcommittee  (or  one  of  its  working  groups)  serve  as  an  advisory  group  for  the 
purpose  of  revisiting  the  development  of  interagency  STEM  education  research  initiatives  in 
specific  areas  of  mutual  interest. 


Attracting  Qualified  Investigators  to  Undertake  Research  in  STEM  Education 


A  number  of  recent  reports  have  lamented  that  education  practice  is  not  firmly  grounded  in  an 
empirical  research  base.  Instead,  policy  decisions  are  oftentimes  based  on  personal  experience, 
folk  wisdom,  and  ideology.  Grounding  education  policy  and  practice  in  the  United  States  on 
evidence  will  require  a  transformation  of  the  field.  Practitioners  will  have  to  turn  routinely  to 
education  research  when  making  important  decisions  and  education  researchers  will  have  to 
produce  research  that  is  relevant  to  those  decisions.  To  achieve  this  goal,  there  is  a  need  for  a 
cadre  of  well-trained  researchers  capable  of  conducting  high  quality  research  that  is  relevant  to 
practitioners  and  policymakers.  To  this  end,  the  Task  Group  recommends  the  following  actions 
to  attract  qualified  students  to  STEM  education  and  evaluation  research  fields:  establish 
innovative  pre-doctoral  and  post-doctoral  training  programs  in  STEM  education  and  evaluation 
research  (the  recent  efforts  of  the  Institute  of  Education  Sciences  Pre-Doctoral  Interdisciplinary 
Research  Training  Program  in  the  Education  Sciences  provide  a  innovative  model  in  this  area); 
establish  an  Early  Career  STEM  Education  Research  grant  program,  with  the  goal  of  funding 
research  carried  out  by  Principal  Investigators  who  are  in  the  early  stages  of  their  careers  (e.g., 
within  5  years  of  having  been  awarded  their  Ph.D.);  and  include  projects  in  the  portfolio  such  as 
centers  that  combine  support  for  graduate  students,  faculty  with  different  kinds  of  expertise,  and 
opportunities  for  collaboration  across  fields  and  institutions. 

Concluding  Observations  and  Future  Directions 

Several  concluding  observations  and  suggestions  regarding  future  directions  are  discussed.  First, 
a  recent  national  survey  indicates  that  many  American  parents  and  students  do  not  believe  that 
more  math  and  science  needs  to  be  taught  in  our  schools.  Consequently,  the  Task  Group  suggests 
that  we  are  in  need  of  evidence-based  approaches  for  doing  a  better  job  of  helping  the  American 
public  become  more  cognizant  of  the  growing  importance  of  STEM  education  for  the  future 
economic  prosperity  of  this  country.  A  second  recommendation  concerns  the  need  for  improved 
methods  of  identifying  and  developing  the  unique  talents  of  students  who  show  early  promise  of 
becoming  scientific  leaders  and  innovators.  Finally,  while  the  Task  Group  acknowledges  the 
important  contributions  of  the  recent  National  Academy  report,  “Rising  Above  the  Gathering 
Storm:  Energizing  and  Employing  America  for  a  Brighter  Economic  Future,”  it  points  out  an 
important  shortcoming  raised  by  the  authors  themselves.  Namely,  although  the  report 
recommends  the  adoption  of  several  existing  K-12  programs,  the  authors  also  note  that  “.  .  .  we 
must  emphasize  the  need  for  research  and  evaluation  to  serve  as  a  foundation  for  change  in  K-12 
mathematics  and  science  education”  (p.  94).  We  wholeheartedly  endorse  the  assertion  that 
research  and  evaluation  can  provide  a  strong  foundation  for  education  reform.  Furthermore,  it  is 
hoped  that  the  analyses  provided  in  the  present  report  will  assist  federal  agencies  in  their 
continuing  efforts  to  strengthen  the  federal  STEM  education  research  portfolio,  so  that  future 
recommendations  for  improving  instructional  practices,  student  achievement,  and  professional 
development  in  STEM  education  will  indeed  be  founded  on  a  rigorous  evidence  base. 
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Introduction 

Formation  of  The  STEM  Education  Research  Task  Group 

The  Education  and  Workforce  Development  Subcommittee  (EWD)  of  the  National  Science  and 
Technology  Council  (NSTC)  was  established  “to  define  the  current  and  future  need  to  attract, 
retain  and  retrain  workers  that  provide  the  critical  Science,  Technology,  Engineering  and 
Mathematics  (STEM)  capabilities  required  within  the  Federal  workforce,  to  identify  and  support 
Federal  efforts  to  contribute  to  the  national  STEM  workforce,  and  to  propose  solutions,  actions 
and  initiatives  that  address  those  needs.”  To  conduct  its  work,  the  EWD  Subcommittee  created 
five  working  groups  that  reflected  the  most  concentrated  interests  common  to  the  agencies 
represented:  1)  Human  Capacity  in  STEM;  2)  Coordination  of  HBCU/HSI/Tribal  Colleges/MSI 
Initiatives;  3)  Federal  R&D  Workforce;  4)  Evaluation;  and  5)  Graduate/  Postdoctoral  Support. 

In  an  effort  to  carry  out  a  more  focused  analysis  of  the  federal  government’s  investment  in 
education  research  within  the  various  STEM  areas,  the  EWD  Subcommittee  decided  to  form  the 
STEM  Education  Research  Task  Group  (affiliated  with  the  Human  Capacity  in  STEM  working 
group).  This  Task  Group  was  comprised  of  representatives  from  the  National  Science 
Foundation,  the  U.S.  Department  of  Education’s  Institute  of  Education  Sciences,  and  the 
National  Institute  of  Child  Health  and  Human  Development  at  the  National  Institutes  of  Health 
(A  list  of  the  Task  Group  members  is  provided  in  Appendix  A). 

The  major  charges  to  this  group  were  to: 

•  Review  and  appraise  the  depth  and  content  of  the  current  federal  investment  in  research 
on  learning  and  education  in  science,  technology,  engineering  and  mathematics  (STEM) 
and  in  evaluation  research,  K-20. 

•  Provide  recommendations  for  strengthening  the  federal  education  research  portfolio. 

About  This  Report 

The  STEM  Education  Research  report  provides  a  review  and  appraisal  of  the  federal  investment 
in  science,  technology,  engineering,  and  mathematics  (STEM)  education  research,  along  with 
recommendations  for  strengthening  the  federal  research  portfolio  in  this  area.  Among  other 
objectives,  a  major  goal  of  the  report  is  to  stimulate  the  development  of  federal  research  agendas 
both  within  and  among  agencies  that  will  lead  to  the  strengthening  of  our  knowledge  base  in  this 
field  in  the  short  run,  and  to  improvements  in  STEM  learning  and  educational  practices  in  the 
long  run.  It  should  also  be  noted  that  the  recommendations  which  have  emerged  from  the  data 
reviewed  in  this  report  can  best  be  interpreted  if  viewed  within  the  broader  context  of  the  EWD 
Subcommittee ’s  purpose  and  scope. 

The  report  is  organized  into  eight  sections.  Section  I  is  a  snapshot  of  the  federal  STEM  education 
research  portfolio,  which  outlines  the  federal  investment  in  STEM  education,  learning,  and 
evaluation  research  in  FY  2003.  Section  II  provides  a  listing  of  the  program  objectives  that  have 
guided  these  investments.  Section  III  describes  how  the  results  of  federally  funded  STEM 
education  research  are  conveyed  to  policymakers  and  practitioners  and  what  is  being  done  to 
close  the  “research  to  practice  gap.”  Section  IV  consists  of  a  synthesis  of  recommendations  that 
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have  been  put  forward  in  STEM  education  and  workforce  reports  issued  over  approximately  a 
10-year  period  preceding  the  writing  of  the  present  report.  These  recommendations,  which  may 
require  additional  study  by  the  research  community,  ultimately  could  lead  to  redirection  of 
investments  by  federal  granting  agencies.  Taken  together,  Sections  I  -  IV  provide  the  basis  for 
identifying  gaps  in  our  knowledge  as  well  as  in  the  federal  research  investment. 

Section  V  examines  how  well  the  current  research  portfolios  of  the  participating  federal  agencies 
are  addressing  the  recommendations  summarized  in  Section  IV.  Section  VI  recommends  several 
ways  in  which  the  participating  federal  agencies  can  work  together  to  increase  efficiency  and 
coordination  of  current  programs  and  cover  the  research  gaps  identified  in  the  current  STEM 
education  and  evaluation  research  portfolios.  Section  VII  suggests  steps  that  can  be  taken  to 
interest  investigators  who  study  other  societal  issues  to  engage  in  research  on  education.  Section 
VIII  provides  examples  of  what  can  be  done  to  attract  additional  students  to  the  education  and 
evaluation  research  fields. 
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Section  I:  Snapshot  of  the  Federal  STEM  Education  Research  Portfolio 

—  FY  2003 

Identifying  Federal  Agencies  That  Invest  in  STEM  Education  Research 

The  first  step  in  reviewing  the  current  federal  investment  required  the  Task  Group  to  identify  the 
federal  agencies  that  invest  in  STEM  education  and  learning  research  and  in  evaluation  research, 
ranging  from  basic  research  on  learning  to  the  study  of  large-scale  organizational  change  within 
the  education  system  from  preK-20.  Identification  of  federal  agencies  supporting  education 
research  was  arrived  at  by  searching  the  Grants.gov  website  as  well  as  individual  federal  agency 
websites,  by  making  inquiries  to  representatives  of  federal  agencies  serving  on  various  STEM 
education  research  committees,  and  by  examining  the  Guidebook  of  Federal  Resources  for  K-12 
Mathematics  and  Science  2004-2005.  The  Task  Group  identified  three  primary  agencies  that 
have  a  significant  and  targeted  focus  on  funding  research  in  STEM  education:  the  U.S. 
Department  of  Education  (ED),  the  National  Institutes  of  Health  (NIH),  and  the  National  Science 
Foundation  (NSF).  (See  Appendix  B  for  more  details  pertaining  to  the  justification  for  not 
including  other  federal  STEM  education  programs  in  this  analysis.) 

Selecting  the  Funding  Programs  and  Projects  to  Be  Studied 

Although  numerous  federal  agencies  support  STEM  education,  most  of  them  do  not  fund 
educational  “research.”  For  the  purposes  of  this  report,  it  was  necessary  to  clearly  define 
education  and  learning  research,  as  well  as  evaluation  research.  Research,  as  defined  for  this 
report,  refers  to  any  systematic  investigation  designed  to  develop  or  contribute  to  the  knowledge 
base  pertaining  to  STEM  education  and  learning.  Excluded  were  demonstration  projects, 
reviews  of  the  literature,  meta-analytic  studies,  and  routine  evaluations  of  curriculum  projects  or 
programs.  However,  evaluation  research  projects  were  included  if  they  qualified  as  compelling 
evaluative  studies  that  build  the  knowledge  base  about  effective  STEM  education  policy  and/or 
practice. 

Also  excluded  were  Small  Business  Innovation  Research  (SBIR)  and  Small  Business 
Technology  Transfer  (STTR)  projects.  However,  it  should  be  noted  that  all  three  federal  agencies 
whose  programs  are  examined  in  this  report  fund  an  extensive  array  of  these  types  of  STEM 
education  studies.  Although  projects  of  this  type  must  incorporate  research  and  evaluation 
components,  the  vast  majority  of  these  studies  are  primarily  designed  to  yield  a  commercial 
product  rather  than  to  contribute  to  the  STEM  education  knowledge  base.  Indeed,  they  often 
build  on  the  knowledge  base  in  an  effort  to  improve  practice.  Furthermore,  collecting  this  vast 
quantity  of  information  would  have  been  prohibitive  given  the  substantial  amount  of  other  data 
that  had  to  be  reviewed  in  order  to  be  appropriately  responsive  to  the  NSTC  Subcommittee’s 
charge.  Therefore,  it  was  decided  that  SBIR  and  STTR  projects  would  not  be  included  in  the 
present  report.  This  decision  should  by  no  means  be  construed  as  a  negative  judgment  of  their 
value  to  the  overall  educational  enterprise.  Indeed,  numerous  policymakers,  educational 
organizations,  and  other  stakeholders  are  extremely  interested  in  the  relationship  between  federal 
STEM  education  programs  and  private  industry,  particularly  with  respect  to  finding  ways  to 
build  on  the  advances  in  educational  technologies  growing  out  of  this  kind  of  work.  As  such,  we 


12 


strongly  recommend  that  this  kind  of  information  be  gathered  and  evaluated  for  inclusion  in  any 
subsequent  STEM  education  research  reports. 

The  Task  Group  decided  that  the  most  reasonable  approach  to  describing  the  Federal  STEM 
education  research  portfolio  was  to  present  a  snapshot  of  the  relevant  programs  and  expenditures 
for  FY  2003.  This  particular  fiscal  year  was  selected  because  at  the  time  the  Task  Group  began 
its  work,  FY  2003  was  the  most  recent  year  for  which  the  requisite  financial  investment  data 
were  available.  Then,  proceeding  according  to  the  definitions  and  inclusion  criteria  described 
above,  the  Task  Group  members  each  culled  their  respective  agency’s  portfolios  and  identified  a 
total  of  29  funding  programs  that  were  operative  in  FY  2003.  As  can  be  observed  in  Table  1,  14 
programs  were  identified  in  ED,  two  programs  in  NIH,  and  13  programs  in  NSF. 
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Table  1.  Federal  Agency  Programs  That  Funded  STEM  Education  Research  in  FY'  2003 


Funding  Agency 


Primary 

Secondary 

Same  of  Research  Program 

EDI 

Preschool  Curriculum  Evaluation  Research  (PCER)  Program 

ED-2 

Teacher  Quality  Research  Grants  (TQR) 

ED-3 

National  Center  for  Improving  Student  Learning  and  Achievement 
in  Mathematics  and  Science 

ED-4 

NIH/NSF 

Interagency  Education  Research  Initiative  (IERI) 

ED-5 

Cognition  and  Student  Learning  (CASL)  Research  Grants 

ED-6 

Mathematics  and  Science  Education  Research  Grants 

ED-7 

Consortium  for  Policy  Research  in  Education  (CPRE)-Study  for 
Instructional  Improvement 

ED-8 

Field  Initiated  Studies  (FIS)  Program 

ED-9 

Development,  Implementation  and  Impact  Evaluation  of  Academic 
Instruction  for  After-School  Programs 

ED-10 

Evaluation  of  the  Effectiveness  of  Educational  Technology 
Intervention 

ED-11 

Research  and  Innovation 

ED-12 

Steppingstones  of  Technology  Innovation 

ED-13 

Research  Institutes  on  Technology 

ED-14 

Research  on  Educational  Captioning 

■ 

NIH-1 

ED/OSERS 

NICHD  Mathematics  &  Sciences  Cognition  and  Learning: 
Development  and  Disorders  (Math) 

NIH-2 

NICHD  Mathematics  &  Sciences  Cognition  and  Learning: 
Development  and  Disorders(Science) 

NSF-1 

Evaluative  Research  and  Evaluation  Capacity  Building  (EREC) 

NSF-2 

ED/N1H 

Interagency  Education  Research  Initiative  (IERI) 

NSF-3 

Various  programs  in  Social  Behavioral  and  Economic  Sciences 
(SBE) 

NSF-4 

Advanced  Technological  Education  (ATE) 

NSF-5 

Course,  Curriculum  and  Laboratory  Improvement  (CCLI), 
Assessment  of  Student  Achievement  Track  (ASA) 

NSF-6 

Research  on  Gender  in  Science  and  Engineering  (GSE) 

NSF-7 

Research  Disabilities  Education  (RDE) 

NSF-8 

Research  on  Learning  and  Education  (ROLE) 

NSF-9 

Science,  Technology,  Engineering  and  Mathematics  Talent 
Expansion  Program  (STEP) 

NSF-IO 

Teacher  Professional  Continuum  (TPC) 

NSF-1 1 

Centers  for  Learning  and  Teaching 

NSF-12 

Instructional  Materials  Development  (IMD)- Applied  Research 
Component 

NSF-13 

Research  Evaluation  and  Technical  Assistance  for  MSP 
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Federal  Agency  STEM  Education  Research  Investment  for  FY  2003 

Table  2  provides  the  approximate  FY  2003  STEM  education  research  investment  by  each  of  the 
three  major  federal  agencies  targeting  the  separate  STEM  areas  as  well  as  collective  STEM 
education  research  activities.  It  must  be  emphasized  here  that  the  dollar  amounts  shown  in  this 
table  represent  agency  estimates  of  the  total  FY  2003  investments  of  the  various  programs  listed 
in  Table  1.  The  procedures  used  by  each  agency  to  arrive  at  these  values  are  provided  below, 
along  with  some  comparative  data.  Supplementary  information  can  be  found  in  Appendix  C. 

Table  2.  Estimated  FY  2003  STEM  Education  Research  Investment  for  Federal  Agency  Programs 


Primary 

Science  Math 

Technology 

Engineering 

Combined 

STEM 

Total 

U.S.  Department  of  Education 

730,000 

8,500,000 

17,700,000 

1,400,000 

28,330,000 

National  Institutes  of  Health 

760,000 

5,240,000 

6,000,000 

National  Science  Foundation 

46,100,000 

52,600,000 

25,500,000 

13,220,000 

18,300,000 

155,720,000 

TOTALS 

47,590,000 

66,340,000 

43,200,000 

13,220,000 

19,700,000 

$190,050,000 

As  shown  in  Table  2,  the  total  estimated  federal  investment  in  STEM  education  and  learning 
research  for  FY  2003  was  approximately  $190  million.  The  breakdown  by  agency  indicates  that 
in  FY  2003,  82  percent  of  funding  for  STEM  education  research  came  from  the  National  Science 
Foundation,  15  percent  from  the  U.S.  Department  of  Education,  and  3  percent  from  the  National 
Institute  of  Child  Health  and  Human  Development,  NIH.  Figure  1  shows  that  35  percent  of  this 
amount  was  directed  toward  research  in  math  education,  25  percent  toward  science,  23  percent 
toward  technology,  7  percent  toward  engineering,  and  10  percent  toward  combined  STEM  areas. 

It  should  be  noted  that  time  constraints  as  well  as  complexities  associated  with  data  collection  for 
all  relevant  programs  precluded  a  comparably  detailed  analysis  of  the  FY  2004  federal  STEM 
education  research  investment.  Likewise,  lack  of  availability  of  pertinent  FY  2005  data  during 
final  preparation  of  this  report  did  not  permit  such  an  analysis  for  that  fiscal  year.  Nevertheless  it 
should  be  pointed  out  that  as  described  below  in  the  agency-specific  sections  of  this  report, 
several  of  the  mathematics  and  science  education  programs  at  the  U.S.  Department  of 
Education’s  Institute  of  Education  Sciences  grew  significantly  during  FYs  2004  and  2005. 
Similarly,  the  first  cohort  of  the  National  Foundation’s  new  Science  of  Learning  Centers  were 
funded  in  FY  2004.  At  the  National  Institutes  of  Health,  the  levels  of  support  for  STEM 
education  and  learning  research  provided  by  the  National  Institute  of  Child  Health  and  Human 
Development  during  FYs  2004  and  2005  were  roughly  comparable  to  that  of  FY  2003. 
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Figure  1.  FY  2003  Education  Research  Investment  by  STEM  Category 


Note:  We  believe  that  all  of  the  education  research  areas  represented  in  this  figure  should  continue  to  be  funded  by 
federal  agencies.  However,  without  a  clear-cut  rationale,  it  would  not  be  appropriate  to  simply  balance  the  level  of 
funding  across  the  identified  areas.  For  example,  it  is  not  surprising  that  technology  and  engineering  education 
research  have  received  less  funding  than  the  math  and  science  areas,  especially  in  the  K-12  arena.  Nonetheless, 
based  in  part  on  the  recommendations  of  the  1997  President’s  Council  of  Advisors  on  Science  and  Technology 
(PCAST)  report,  it  is  clear  that  the  federal  government  has  the  obligation  to  invest  in  technology,  both  as  an  aid  to 
learning  and  as  a  means  of  collecting  research  data  about  learning.  Technology  has  been  supported  over  the  years 
since  the  PCAST  report  was  issued,  as  is  evident  in  the  IERI  portfolio.  However,  other  than  NSF-4,  there  have  been 
few  federal  programs  or  initiatives  aimed  specifically  at  research  on  the  technological  workforce.  Instead,  initiatives 
such  as  IERI  and  the  FY  2005  NSF  initiative  on  Advanced  Learning  Technologies  have  funded  technology  in  the 
service  of  science,  math  and  engineering  learning.  For  similar  reasons,  engineering  education  research  has  been 
funded  at  a  lower  level  than  math  and  science,  primarily  because  few  engineering  programs  exist  at  the  K-12  level. 
However,  the  NSF  Engineering  Directorate  is  moving  to  Rind  much  more  research  on  undergraduate  engineering  in 
its  Division  of  Engineering  Education  and  Centers. 
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U.S.  Department  of  Education.  Within  the  U.S.  Department  of  Education  in  FY  2003,  there  were 
two  primary  sponsors  of  STEM  education  and  learning  research:  The  Institute  of  Education 
Sciences  (IES)  (ED-1  through  ED-10  in  Table  1),  and  the  Office  of  Special  Education  and 
Rehabilitative  Services  (OSERS)  (ED-1 1  through  ED-14).  IES  was  established  in  FY  2003.  The 
IES  budget  is  comparatively  small.  Specifically,  in  FY  2003,  the  research  and  dissemination 
account  of  $139.1  million  and  the  statistics  account  of  $89.4  million  together  constituted  less 
than  one  half  of  one  percent  of  the  $50.2  billion  discretionary  budget  of  the  U.S.  Department  of 
Education  (Whitehurst,  2003).  However,  some  IES  STEM  educational  evaluation  research 
activities,  specifically  those  conducted  by  the  National  Center  for  Education  Evaluation  and 
Regional  Assistance  in  IES  (ED-9  and  ED- 10  in  Table  1),  are  supported  by  federal  funds  set 
aside  or  authorized  only  for  evaluation  studies  through  Title  I  and  Title  II  of  the  Elementary  and 
Secondary  Education  Act.  At  the  time  of  data  collection  for  this  report,  the  IES  research 
competitions  were  just  coming  online.  As  a  result,  only  a  limited  number  of  projects  were  funded 
in  FY  2003.  These  programs  have  grown  significantly  since  that  time.  For  instance,  the  total 
dollars  awarded  (i.e.,  total  amount  for  the  grant,  including  out-years)  for  projects  under  the 
Mathematics  and  Science  Education  Research  Grant  Program  increased  from  $1.4  million  in  FY 
2003  to  $23.6  million  in  FY  2005.  As  interest  in  IES  research  programs  has  grown,  new 
programs  have  been  added.  For  example,  in  FY  2005  the  Teacher  Quality  research  program  (ED- 
2)  was  divided  into  two  separate  competitions:  Teacher  Quality  -  Mathematics  and  Science 
Education,  and  Teacher  Quality  -  Reading  and  Writing.  Finally,  with  the  addition  of  the  National 
Center  for  Special  Education  Research  to  IES,  new  research  programs  (e.g.,  Mathematics  and 
Science  Special  Education  Research  Program)  have  been  created  to  support  research  on  the 
development  and  evaluation  of  mathematics  and  science  curricula,  instructional  practices,  and 
assessments  for  improving  learning  and  achievement  for  students  with  identified  disabilities  and 
students  at  risk  for  disability. 

National  Institutes  of  Health.  The  National  Institute  of  Child  Health  and  Human  Development 
(NICHD)  at  the  National  Institutes  of  Health  (NIH),  has  funded  selective  research  efforts  over 
the  past  20  years  pertaining  to  the  learning  of  mathematics  and  science.  However,  in  FY  2003, 
the  Institute  formally  established  the  Mathematics  and  Science  Cognition  and  Learning  program, 
designed  specifically  to  support  research  on  both  nonnal  and  atypical  development  of  math  and 
science  learning.  This  program  (see  portfolios  NIH-1  and  NIH-2  in  Table  1)  constitutes  one  of 
seven  different  programs  comprising  the  Child  Development  and  Behavior  Branch  (CDBB)  at 
NICHD.  The  CDBB  supports  research  and  research  training  relevant  to  the  psychological, 
psychobiological,  language,  behavioral,  and  educational  development  of  children.  Four  of  the 
Branch’s  seven  programs  fund  learning  and  educational  research,  including:  a)  Human  Learning 
and  Learning  Disabilities;  b)  Language,  Bilingual,  and  Biliteracy  Development  and  Disorders 
and  Adult,  Family,  and  Adolescent  Literacy;  c)  Early  Learning  and  School  Readiness;  and  d) 
Mathematics  and  Science  Cognition  and  Learning  —  Development  and  Disorders.  The  FY  2003 
research  budget  for  these  four  CDBB  programs  was  approximately  $35.9  million  (i.e.,  includes 
research  and  center  grants  only,  and  excludes  contracts  as  well  as  SBIR,  training,  career,  and 
fellowship  grants;  it  should  also  be  noted  that  not  all  of  the  research  grants  in  these  programs 
pertain  to  learning  or  education  research).  Consequently,  the  FY  2003  budget  for  research 
projects  in  the  NICHD  Mathematics  and  Science  Cognition  Program  (i.e.,  portfolios  NIH-1  and 
NIH-2  in  Table  1),  approximately  $6  million,  thus  amounted  to  roughly  17  percent  of  the 
CDBB’s  FY  2003  expenditures  in  the  areas  of  learning  and  education  research.  Finally,  the 
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levels  of  support  provided  by  this  program  for  math  and  science  cognition  and  learning  research 
during  FYs  2004  and  2005  were,  by  and  large, comparable  to  that  of  FY  2003. 

The  National  Science  Foundation.  In  FY  2003,  the  National  Science  Foundation  had  a  total 
budget  request  of  approximately  $5.03  billion,  of  which  $3.8  billion  was  designated  for  the 
research  and  related  activities  directorates  and  $0.91  billion  for  the  Education  and  Human 
Resources  (EHR)  Directorate,  where  the  bulk  of  education  research  and  evaluation  research  are 
funded.  The  approximately  $155  million  used  by  NSF  to  support  education  research  in  2003  thus 
represents  about  17%  of  the  EHR  budget  and  4%  of  the  total  NSF  budget.  However,  all  of  the 
other  NSF  directorates  (Biological  Sciences;  Computer  and  Information  Science  and 
Engineering;  Engineering;  Geosciences;  Mathematical  and  Physical  Sciences;  Social,  Behavioral 
and  Economic  Sciences)  and  the  Office  of  Polar  Programs  fund  K-12  as  well  as  post-secondary 
education  projects,  and  many  of  them  invest  in  education  research.  We  estimate  that  their 
investment  in  education  research  would  add  approximately  $20  million  to  the  NSF  total  for  FY 
2003.  In  addition,  because  this  analysis  comprises  a  snapshot  of  only  the  2003  investment,  it 
does  not  include  such  new  NSF  programs  as  the  Science  of  Learning  Centers,  the  first  cohort  of 
which  (four  centers)  were  funded  in  FY  2004  at  the  level  of  $5  million  each  per  year  for  five 
years. 
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Section  II:  Federal  Agency  Program  Objectives  for  STEM  Education 

Research 


Developing  the  Categories  of  Objectives 

Table  3  includes  the  categories  and  objectives  guiding  current  investment  in  STEM  education 
research  by  the  U.  S.  Department  of  Education,  the  National  Institutes  of  Health,  and  the 
National  Science  Foundation.  These  categories  were  developed  by  the  Task  Group  after 
analyzing  four  principal  sources: 

•  The  Research  Sub-group  Synthesis  of  the  U.S.  Department  of  Education’s  Mathematics 
and  Science  Initiative; 

•  The  National  Science  Foundation’s  Portfolio  Review  of  mathematics  education  projects: 
Project  Jacket  Review  Questionnaire  prepared  by  the  Urban  Institute; 

•  The  ERIC  Clearinghouse  Controlled  Vocabulary  Thesaurus;  and 

•  The  U.S.  Department  of  Education,  Institute  of  Education  Science’s  abstract  descriptors. 

First,  a  comprehensive  list  of  terms  was  compiled  from  these  sources.  Then,  after  ordering  them, 
the  Task  Group  reduced  and  refined  them  so  that  clear  distinctions  could  be  made  among  the 
tenns. 

Collecting  and  Analyzing  the  Program  Data 

From  the  outset,  it  was  detennined  that  the  unit  of  analysis  would  be  the  federal  research 
programs,  rather  than  the  individual  projects  funded  under  the  programs.  The  Task  Group 
reviewed  published  solicitations,  program  announcements,  and  program  descriptions  to 
determine  the  intent  of  the  programs’  prospective  investments.  To  this  end,  a  datasheet  was 
devised  to  collect  quantitative  and  qualitative  data  on  32  identified  federal  agency  STEM 
education  research  programs.  Data  were  collected  in  the  following  areas: 

•  Specific  program  objectives 

•  Specific  program  expectations  (policy  questions  and  social  impact) 

•  Research  methods  preferred  by  the  specific  program 

•  How  the  results  of  the  funded  research  are  conveyed  to  policymakers 

•  How  the  results  of  the  funded  research  are  conveyed  to  practitioners 

Staff  at  each  federal  agency  gathered  comprehensive  information  about  the  programs  and 
provided  electronic  files  to  a  designated  central  contact  on  the  Task  Force.  The  qualitative  data 
were  then  disaggregated  into  unique  elements,  which  were  essentially  individual  statements  of 
program  objectives.  Each  data  element  was  assigned  an  agency  program  code  and  a  categorical 
keyword.  Finally,  the  data  were  checked  for  accuracy  and  edited  for  consistency  of  expression. 

Two  qualitative  data  tables  were  generated:  one  organized  by  the  objectives  category  and  another 
by  federal  program  (see  appendices  C  and  D,  respectively).  These  tables  were  analyzed 
thematically  for  use  in  this  report.  (It  should  be  noted  that  the  highlighted  phrases  in  these  tables 
represent  priority  statements  which  address  multiple  topics  and  thus  could  be  placed  in  more  than 
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one  category.  The  highlighting  should  assist  the  reader  in  ascertaining  why  that  statement 
received  its  designated  classification.) 

Table  3  lists  the  STEM  education-related  research  programs  in  ED,  NIH,  and  NSF  that  were 
originally  examined  with  respect  to  their  FY  2003  investment.  However,  it  should  be  noted  that 
this  table  provides  information  about  the  objectives  of  these  programs  that  guide  their  FY  2005 
investments.  Consequently,  in  comparison  with  the  Table  1  list,  the  reader  will  note  a  name 
change  for  the  ED-8  program,  the  addition  of  two  ED  programs  (ED- 15  and  ED- 16)  that  were 
initiated  after  FY  2003,  and  the  addition  of  one  NSF  program  (NSF- 14).  An  “X”  in  any  given 
cell  signifies  that  the  category  for  that  column  is  currently  an  objective  of  the  program  indicated 
by  that  row.  For  example,  “evaluation  and  assessment”  is  a  priority  of  the  U.S.  Department  of 
Education’s  “Preschool  Curriculum  Evaluation  Research  (PCER)”  program.  A  brief  rationale  for 
each  agency’s  choices  for  inclusion  in  Table  3  can  be  found  in  Appendix  B.  Appendix  C 
contains  all  program  objectives  sorted  by  category  (e.g.,  “capacity  building”)  along  with  the 
published  program  authority  for  each  objective  (taken  from  requests  for  proposals,  program 
announcements,  etc.).  Appendix  D  contains  the  same  information,  sorted  by  program  (e.g.,  all 
program  objectives  for  NSF  program  #11,  Centers  for  Learning  and  Teaching,  are  listed 
together). 

Programs  Objectives  -  Data  Summary 

As  can  be  observed  in  Table  3,  most  of  the  federal  STEM  education  research  objectives  focus  on 
curriculum  or  instructional  practice,  student  cognition  and  learning,  and  evaluation  and 
assessment.  Fourteen  programs  include  various  equity  issues  in  their  research  objectives,  and  for 
the  most  part,  these  programs  appear  to  be  specifically  focused  on  equity  issues  (e.g.,  ED-1 1, 
ED-12,  ED-13,  ED-14,  NIH-1,  NIH-2,  NSF-6,  and  NSF-7).  Higher  education,  Faculty 
development  and  STEM  workplace  improvement  are  covered  only  by  NSF. 

One  of  the  charges  to  the  Task  Group  was  to  review  the  various  types  of  research  methodologies 
recommended  by  the  federal  agencies.  Rather  than  embedding  this  information  in  Table  3,  we 
chose  to  provide  a  more  detailed  description  of  these  approaches  as  outlined  below. 
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Table  3.  Categories  of  Objectives  That  Guide  Current  Investments  of  Federal  STEM  Education  Research  Programs 


PROGRAM 

CODE 

Capacity 
Building  for 

STEM  ED 

Research 

Curriculum 

or 

Instructional 

Practice 

Equity 

Evaluation  & 

Assessment 

Faculty 

Development 

Schoolwide 

& 

Systemwide 

Reforms 

STEM 

Education 

Policies 

STEM 

Workforce 

Improvement 

Student 

Cognition  & 

Learning 

Teacher 

Development 

Disabilities 

Gender 

Learning 

Difficulties 

Race/ 

Ethnicity 

SES 

ED-1 

X 

X 

X 

X 

ED-2 

X 

X 

X 

X 

ED-3 

X 

X 

X 

EDA 

X 

X 

X 

ED-5 

X 

X 

ED-6 

X 

X 

X 

X 

ED-7 

X 

X 

X 

X 

X 

X 

ED-8 

X 

ED-9 

X 

X 

ED- 10 

X 

X 

ED- 11 

X 

X 

X 

X 

ED-12,13,14 

X 

X 

ED- 15 

X 

ED- 16 

X 

NIH-1 

X 

X 

X 

X 

X 

X 

NIH-2 

X 

X 

X 

NSF-1 

X 

X 

X 

NSF-2 

X 

X 

X 

X 

X 

NSF-3 

X 

NSF-4 

X 

X 

X 

X 

X 

NSF-5 

X 

X 

X 

NSF-6 

X 

X 

X 

X 

NSF-7 

X 

X 

X 

NSF-8 

X 

X 

X 

NSF-9 

X 

X 

NSF-10 

X 

NSF-1 1 

X 

X 

X 

X 

X 

X 

X 

NSF-12 

X 

X 

X 

X 

NSF-13 

X 

X 

X 

X 

X 

NSF-14 

X 

X 

X 

X 

X 

X 

X 

X 

US  Department  of  Education 

ED-1  Preschool  Curriculum  Evaluation  Research  (PCER)  Program 

ED-2  Teacher  Quality  Research  Grants  (TQR) 

ED-3  National  Center  for  Improving  Student  Learning  &  Achievement  in  Math  &  Science(closed  2/2004) 
ED-4  Interagency  Education  Research  Initiative  (IERI) 

ED-5  Cognition  &  Student  Learning  (CASL)  Research  Grants 
ED-6  Mathematics  &  Science  Education  Research  Grants 

ED-7  Consortium  for  Policy  Research  in  Education  Study  of  Instructional  Improvement 
ED-8  Field  Initiated  Evaluations  of  Education  Interventions  (formerly  Field  Initiated  Studies) 

ED-9  Development,  Implementation  &  Eval  of  Academic  Instruction  for  After-School  Prog 

ED- 10  Evaluation  of  the  Effectiveness  of  Educational  Technology  Intervention 

ED- 11  Research  and  Innovation 

ED- 1 2  Steppingstones  of  T echnology  Innovation 

ED- 1 3  Research  Institutes  on  T echnology 

ED- 14  Research  on  Educational  Captioning 

ED-15  Pre-Doctoral  Interdisciplinary  Research  Training  Fellowship  Program 
ED- 16  Post-Doctoral  Research  Training  Program 


National  Institutes  of  Health 

NIH-1  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Math 

NIH-2  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Science 

National  Science  Foundation 

NSF-1  Evaluative  Research  &  Evaluation  Capacity  Building  (EREC) 

NSF-2  Interagency  Education  Research  Initiative 

NSF-3  Various  programs  in  Social  Behavioral  and  Economic  Sciences  (SBE) 

NSF-4  Advanced  Technological  Education  (ATE) 

NSF-5  Course,  Curriculum  &  Laboratory  Improvement/Assessment  of  Student  Achievement 
NSF-6  Research  on  Gender  in  Science  &  Engineering  (GSE) 

NSF-7  Research  Disabilities  Education  (RDE) 

NSF-8  Research  on  Learning  &  Education  (ROLE) 

NSF-9  Science,  Technology,  Engineering  and  Mathematics  Talent  Expansion  Program  (STEP) 
NSF-10  Program  Teacher  Professional  Continuum  (TPC) 

NSF-1 1  Centers  for  Learning  and  Teaching 

NSF-12  Instructional  Materials  Development  (IMD)-Applied  Research  Component 
NSF-1 3  Research,  Evaluation  and  Technical  Assistance  (RET A) 

NSF-14  Science  of  Learning  Centers  (SLC) 
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STEM  Learning  and  Educational  Research  Methods:  Federal  Agency  Practices 

During  the  past  several  years,  issues  concerning  the  appropriateness  of  research  designs  and 
methods  have  become  especially  prominent  in  the  educational  research  community  as  well  as  in 
educational  policy  circles.  Increased  interest  in  this  topic  was  stimulated  in  part  by  the  emphasis 
in  the  No  Child  Left  Behind  Act  of  2001  (NCLB)  on  “scientifically-based  research”  and 
“evidence-based”  approaches  to  educational  practice.  In  response,  a  series  of  articles  began  to 
appear  in  scholarly  journals  devoted  to  this  subject  area,  as  well  as  papers  and  symposia 
presented  at  national  conferences  (e.g.,  the  American  Educational  Research  Association).  In 
2002  the  National  Research  Council  published  a  report  examining  the  concept  of  scientific 
research  in  education  ( Scientific  Research  in  Education,  2002).  The  specific  foci  of  this  broad 
range  of  publications  and  presentations  have  included:  philosophical  and  methodological  issues 
(i.e.,  concern  with  the  logical  basis  of  empirical  inquiry),  the  delineation  of  an  assortment  of 
research  methods  and  designs  and  their  applicability  to  different  kinds  of  research  questions, 
high-stakes  assessment,  data  analytic  strategies,  standards  for  evaluation  research,  and  the 
development  of  novel  approaches  for  assessing  fidelity  of  implementation  of  promising 
educational  interventions  when  taken  to  scale. 

Several  of  these  articles  and  reports  have  put  forth  recommendations  to  the  relevant  federal 
agencies  pertaining  to  the  funding  of  projects  that  incorporate  various  types  of  educational 
research  methods.  It  is  important  to  note  that  representatives  from  these  agencies  have  been 
actively  involved  in  national  discussions  surrounding  research  methodologies,  both  to  listen  to 
suggestions  from  the  research  community  and  to  describe  and  explain  their  evolving  positions 
with  respect  to  such  matters.  Toward  this  end,  the  following  sections  characterize  the  objectives 
and  preferences  of  the  relevant  federal  agencies  regarding  the  use  of  methods,  designs,  and 
analysis  in  STEM  learning  and  educational  research.  Even  a  cursory  reading  of  these 
descriptions  reveals  one  commonality  across  the  agencies  -  that  the  research  method(s)  employed 
should  fit  the  question(s)  at  hand  -  which  is  also  consistent  with  one  of  the  recommendations  put 
forward  in  a  recently  published  National  Academy  report,  Advancing  Scientific  Research  in 
Education  (2005).  Beyond  this  guideline,  the  agencies  differ  to  some  extent  in  their  preferences 
for  various  approaches  to  STEM  educational  research.  At  the  very  least,  it  is  equally  clear  that  a 
wide  range  of  approaches  is  deemed  acceptable  by  one  or  more  of  these  agencies. 

Before  describing  the  specific  educational  and  learning  research  designs  and  methods  espoused 
by  each  agency,  it  is  important  to  point  out  that  the  evaluation  or  research  component  of  the  grant 
proposal  is  only  one  element  in  the  overall  peer  review  process — albeit  a  very  crucial  one.  Of 
comparable  import,  however,  is  the  instructional  intervention  or  educational  program  (e.g.,  the 
technology  being  introduced,  the  professional  development  experience,  the  curriculum 
development  activity,  the  underrepresented  enhancement  initiative,  etc.).  In  such  cases,  the 
educational  research  component  or  evaluation  plan  is  an  integral  feature  of  the  overall  project, 
frequently  used  for  the  purpose  of  establishing  the  extent  to  which  the  proposed  activity  was 
effective. 

Institute  of  Education  Sciences 

All  grant  applications  submitted  to  IES  are  peer  reviewed  and  evaluated  for  their  scientific  merit 
based  on  the  following  criteria:  significance,  research  plan,  personnel,  and  resources.  The 


22 


position  of  the  Institute  of  Education  Sciences  is  that  the  methods  employed  in  a  research  project 
must  be  appropriate  to  the  question  being  addressed  and  that  no  single  method  characterizes 
good  science.  Rather  it  is  the  degree  of  match  between  the  method,  the  question,  and  the 
conclusion  that  is  at  issue.  The  methods  supported  by  the  Institute  include  methods  for  producing 
sound  descriptive  summaries,  including  surveys,  observational  data,  and  administrative  records; 
methods  appropriate  for  isolating  possible  relationships  such  as  multivariate  analysis;  and 
methods  designed  to  address  questions  concerning  the  effectiveness  of  particular  policies  or 
practices,  including  single-subject,  quasi-experimental,  and  experimental  approaches. 

The  Institute  strongly  prefers  randomized  field  trials  when  the  question  is  the  effectiveness  of 
mature  programs  and  practices.  Such  trials  virtually  always  include  the  collection  of  process  data 
that  can  provide  insight  into  why  an  intervention  does  or  does  not  work  and  that  allow  an 
examination  of  the  relationship  between  implementation  and  outcomes.  However,  randomized 
trials  are  only  a  part  of  the  Institute's  portfolio.  A  substantial  portion  of  the  Institute's  funding 
goes  to  upstream  work  in  which  researchers  are  developing  new  programs  or  identifying 
promising  practices  using  methods  appropriate  for  those  investigations,  such  as  the  methods 
described  below.  IES  also  invests  in  the  development  and  validation  of  measurement  and 
assessment  tools.  All  of  the  Institute's  research  programs  are  embedded  in  practice,  requiring  the 
selection  of  topics  that  are  highly  relevant  to  practitioners  and  the  conduct  of  research  in 
authentic  education  delivery  settings,  including: 

1.  Descriptive  questions  address  the  state  of  education  (e.g.,  how  are  students  performing,  how 
much  are  teachers  being  paid).  Appropriate  methods  include  large-scale  assessments,  surveys, 
analyses  of  institutional  records,  coded  observations  of  classroom  interactions,  structured 
interviews,  and  case  studies,  among  others. 

2.  Association  questions  address  the  statistical  connections  between  education  conditions  and 
education  outcomes  (e.g.,  how  are  children’s  preschool  abilities  related  to  their  elementary 
school  achievement).  Appropriate  methods  range  from  computing  simple  correlations  between 
two  variables  to  mathematically  sophisticated  statistical  models  for  simultaneously  considering 
the  associations  among  many  conditions  and  variables. 

3.  “What  works”  questions  address  the  causal  effects  of  programs,  practices,  and  approaches  on 
education  outcomes  (e.g.,  does  a  reduction  in  class  size  improve  student  achievement,  does  a 
specific  mathematics  curriculum  produce  gains  in  student  learning?).  IES  has  taken  the  position 
that  randomized  trials,  in  which  participants  are  randomly  assigned  to  condition,  are  the  best 
method  for  determining  what  works.  The  mean  difference  between  outcomes  for  participants  in 
conditions  being  compared  in  well-conducted  randomized  trials  is  an  unbiased  estimate  of  the 
effects  of  those  treatments.  In  circumstances  in  which  a  randomized  trial  is  not  possible, 
alternatives  that  substantially  minimize  selection  bias  or  allow  it  to  be  modeled  may  be  employed 
(e.g.,  appropriately  structured  regression-discontinuity  designs). 

4.  “Why”  questions  address  the  underlying  mechanisms  and  processes  by  which  causal  effects 
occur  (e.g.,  Why  does  one  mathematics  curriculum  produce  better  learning  outcomes  for  middle 
school  students  than  another  mathematics  curriculum?  Is  it  because  of  the  sequencing  of 
components,  the  variation  and  degree  of  transfer  required  in  the  practice  problems,  or  the  amount 
of  explicit  instruction  and  scaffolding  provided?).  Methods  appropriate  to  answering  why 


questions  include  many  of  the  methods  exemplified  in  the  previous  discussion  of  descriptive, 
correlational,  and  causal  questions. 


23 


National  Institutes  of  Health  -  National  Institute  of  Child  Health  and  Human  Development 

Research  grant  applications  submitted  to  the  NICHD/NIH  are  peer  reviewed  and  judged  in  terms 
of  their  scientific  and  technical  merit  in  relation  to  five  criteria:  significance,  approach, 
innovation,  investigators,  and  enviromnent.  However,  an  application  does  not  need  to  be  strong 
in  all  categories  to  be  judged  likely  to  have  major  scientific  impact  and  thus  deserve  a  high 
priority  score  (see  the  following  for  recently  updated  specifics  regarding  the  NIH  criteria  for 
evaluating  research  grants:  http://grants.nih.gov/grants/guide/notice-files/NOT-OD-Q5- 
002.html).  With  respect  to  the  “approach”  criterion,  research  grant  applications  submitted  to  the 
NICHD/NIH  Child  Development  and  Behavior  Branch  can  incorporate  a  wide  variety  of  study 
designs,  research  methods,  and  data-analytic  procedures.  The  most  important  consideration  is  an 
appropriate  fit  of  the  proposed  methodology  to  the  research  question.  Projects  may  incorporate 
experimental,  quasi-experimental,  and  correlational  designs,  among  others.  Of  special  interest  is 
the  use  of  prospective  longitudinal  designs  for  examining  developmental  changes  in 
mathematical  and  scientific  thinking  and  reasoning.  Epidemiological  studies  are  sought  for  the 
purpose  of  estimating  the  prevalence  of  specific  learning  disabilities  in  mathematics.  The 
Institute  also  encourages  the  use  of  assorted  neuroimaging  modalities  (e.g.,  fMRI, 
Magnetoencephalography)  for  investigating  the  neurobiological  substrates  of  both  nonnal  and 
atypical  mathematical  and  scientific  thinking.  Various  types  of  quantitative  genetic  designs  (e.g., 
twin  studies)  may  be  used  for  examining  the  contribution  of  genetic  and  environmental  factors  to 
the  variation  in  mathematical  abilities.  A  variety  of  data-analytic  strategies  may  be  employed, 
driven  in  large  part  by  the  nature  of  the  study  design.  An  analytic  strategy  of  particular  interest  is 
the  use  of  growth  curve  modeling  for  studying  subtype  and  other  individual  differences  in  the 
developmental  trajectories  of  mathematical  learning  skills.  Qualitative  research  approaches  are 
welcomed  as  an  adjunct  to  quantitative  approaches  insofar  as  they  are  methodologically  rigorous, 
that  the  data  they  generate  can  be  considered  trustworthy,  credible,  and  transferable,  and  that 
they  can  be  used  to  enrich  and  illuminate  theoretically  based  interpretations  of  both  the 
qualitative  and  quantitative  data.  Finally,  small-  and  large-scale,  evidenced  based,  instructional 
interventions  may  be  carried  out  so  long  as  methods  suitable  for  testing  efficacy  and/or 
effectiveness  are  employed. 

National  Science  Foundation  -  Directorate  for  Education  and  Human  Resources 

The  Education  and  Human  Resources  (EHR)  Directorate  at  the  National  Science  Foundation 
(NSF)  funds  research  through  all  of  its  divisions  and  major  programs.  All  proposals  submitted  to 
NSF  are  reviewed  by  peer  review  panels  that  are  asked  to  comment  on  each  application's 
intellectual  merit  (for  research  proposals,  this  includes  the  study  design)  and  potential  for  broader 
impact.  All  study  designs  are  appropriate  for  review,  and  EHR  agrees  with  the  authors  of 
Scientific  Research  in  Education  (National  Research  Council,  2002)  that  the  research  questions 
should  drive  the  methods  used  and  that  evidence  must  be  linked  to  theories  or  hypotheses 
through  a  clear  chain  of  reasoning.  Further,  the  best  methodological  approach  depends  on  the 
state  of  knowledge  in  a  particular  area,  with  descriptive  studies  and  design  studies  often  most 
appropriate  for  projects  where  there  is  a  limited  knowledge  base  and  experimental  and  quasi- 
experimental  studies  often  most  appropriate  when  the  knowledge  base  is  more  mature.  The  major 
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research  program  in  EHR  is  Research  on  Learning  and  Education  (ROLE).  At  the  October  2004 
Principal  Investigator  (PI)  meeting  for  ROLE,  over  200  ROLE  Pi’s  responded  to  a  survey 
asking,  among  other  things,  what  methods  they  employ  to  address  their  research  questions.  PI 
responses  identified  the  most  frequently  used  methods  as: 

1 .  Descriptive  case  study  (49%) 

2.  Quasi-Experimental  (36%) 

3.  Surveys  (35%) 

4.  Experimental  Designs  (31%) 

5 .  Design  Experiments  (31%). 

The  percentages  add  up  to  more  than  100%  because  the  majority  of  projects  employ  more  than 
one  methodological  approach.  In  addition,  less  frequently  used  methods  cited  were  research 
syntheses,  ethnographic  studies,  action  research,  meta-analysis,  and  historical  studies.  EHR's 
other  large  research  program,  the  Interagency  Education  Research  Initiative  (IERI),  was 
designed  to  investigate  issues  of  “going  to  scale”  around  interventions  that  have  some  evidence 
base.  In  this  program,  large  studies  have  been  required  to  utilize  quasi-experimental  or 
experimental  designs.  (See  p.  35  for  a  more  thorough  description  of  IERI  and  its  current  status.) 
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Section  III:  Conveying  STEM  Education  Research  Results  to  Policymakers 

and  Practitioners 

Closing  the  Gap  Between  Education  Research  and  Education  Policy 

Too  often,  education  research  does  not  inform  education  policy  at  the  local  or  state  level.  For 
example,  Corcoran  (2003)  examined  the  degree  to  which  central  office  staff  in  three  urban 
school  districts  used  research-based  evidence  to  make  policy  decisions  to  improve  student 
performance,  and  found  that: 

•  Policymakers  tend  to  make  decisions  based  on  political  expediency. 

•  Decision-making  patterns  tend  toward  personal  philosophy  rather  than  evidence. 

•  Policymakers  find  it  hard  to  distinguish  among  empirical  research,  theories,  and  simple 
advocacy. 

At  the  national  level,  Congressional  policymakers  concerned  with  school  improvement  want  to 
know  “what  works”  and  “what  doesn’t  work”  in  elementary  and  secondary  schools  so  that  they 
can  better  focus  program  efforts  and  tax  dollars.  But  this  is  often  an  unrealistic  expectation  as  it 
may  take  many  years  and  multiple  studies  to  bring  research  knowledge  from  questions  about 
what  is  happening  to  questions  that  address  why  or  how  something  is  happening  (National 
Research  Council,  2002).  Research  findings  can  suggest  policy  options  or  components  of  a  local 
response  to  a  problem,  but  determining  what  will  work  best  in  a  particular  district  requires  the 
knowledge  of  its  existing  commitments,  history,  resources,  and  political  climate.  In  other  words, 
one  size  does  not  fit  all,  and  findings  have  to  be  applied  with  careful  thought  to  the  specific 
conditions  or  demographics  of  the  problem  at  hand  (Corcoran,  2003). 

Two  major  barriers  frequently  prevent  policymakers  from  drawing  upon  research  in  arriving  at 
their  judgments  and  decisions: 

•  Existing  research  is  inadequate  to  the  question  at  hand. 

•  Existing  research  is  inconclusive  or  confusing  (e.g.,  conflicting  research  findings) 

In  order  to  overcome  these  barriers,  MacColl  and  White  (1996)  suggested  making  research  data 
more  useful  to  people  based  on  common  sense.  For  example,  they  recommended  that  when 
reporting  study  results  to  non-technical  audiences,  the  findings  should  be  summarized  in  plain 
language  at  the  beginning  of  the  report  so  that  readers  without  a  background  in  research  or 
statistics  can  readily  understand  the  content  of  a  report.  MacColl  and  White  also  argue  that 
researchers  may  help  policymakers  more  clearly  understand  the  process  of  research  by 
discussing  the  kinds  of  questions  that  can  and  cannot  be  answered  using  their  methods  and  by 
clarifying  that  results  are  seldom  definitive. 

How  Federal  Educational  Research  Programs  Disseminate  Research  Findings  to 
Policymakers 

The  Task  Group’s  examination  of  the  ways  in  which  ED,  NIH  and  NSF  disseminate  findings 
from  their  research  revealed  that,  for  the  most  part,  the  results  of  research  studies  are  made 
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available  to  policymakers  (and  any  other  interested  party)  through  the  federal  agency’s  own 
website,  the  projects’  websites,  an  agency  clearinghouse,  and  through  government  reports  such 
as  GPRA  reports.  Professional  conferences  and  scholarly  journals  are  the  next  most  commonly 
used  avenues  for  making  such  findings  available  to  the  public  and  policymakers.  Several 
programs  strongly  encourage  the  recipients  of  their  grants  to  disseminate  findings  via 
professional  conferences  and  scholarly  publications.  In  general,  these  are  considered  passive 
modes  of  delivery  because  policymakers,  educators,  and  the  general  public  must  make  the  effort 
to  find  and  access  research  results. 

In  some  instances,  Federal  programs  deliver  findings  directly  to  policymakers  via  policy  briefs, 
testimony  to  congressional  subcommittees,  meetings  with  policy  experts,  or  seminars.  For 
example  one  program  stated  that  policy  analysts  access  infonnation  about  the  grants  to  prepare 
special  reports  -  but  this  does  not  appear  to  be  a  program-directed  activity.  Several  programs 
stated  that  staff  members  convey  findings  to  policymakers.  Another  indicated  that  study  results 
which  merit  national  attention  are  released  to  the  popular  press.  In  another  instance,  it  was 
reported  that  getting  research  findings  into  the  hands  of  policymakers  is  not  an  explicit  focus. 
Some  grantees,  such  as  institutions  of  higher  education,  initiate  policy  discussions  that 
incorporate  research  results,  but  since  this  activity  tends  to  be  project  rather  than  program- 
directed,  the  Task  Group  was  unable  to  track  the  extent  or  impact  of  their  efforts. 

What  can  federal  educational  research  programs  do  to  better  integrate  research  with 
policy? 

A  number  of  possible  action  steps  have  been  proposed  regarding  how  federal  agencies  might 
improve  the  integration  of  research  and  policy: 

•  Support  periodic  research  syntheses,  such  as  meta-analytic  techniques,  in  critical  areas  so 
that  the  knowledge  base  can  be  made  accessible  to  local  decision-makers  (Corcoran, 
2003).  Although  federal  agencies  have  funded  some  research  syntheses,  for  example, 
studies  conducted  by  the  National  Academy  of  Sciences,  the  redirecting  of  investments  to 
support  additional  efforts  of  this  type  may  well  prove  beneficial.  Consistent  with  this 
view,  a  recent  National  Academy  report  funded  by  the  National  Science  Foundation 
evaluated  the  quality  of  evaluation  studies  of  K-12  mathematics  curricula  (see  Report  #58 
in  Appendix  E).  According  to  the  report,  the  vast  majority  of  the  studies  that  were 
examined  fell  short  of  the  scientific  standards  required  for  gauging  the  overall 
effectiveness  of  the  curricula  they  were  designed  to  evaluate.  Clearly,  intervention  study 
designs  as  well  as  evaluation  approaches  and  criteria  must  meet  more  rigorous  standards 
before  research  syntheses  can  be  appropriately  employed  in  order  to  yield  the  kinds  of 
information  needed  to  improve  educational  policymaking. 

•  Provide  research  in  forms  that  are  easily  digestible  by  non-researchers.  One  approach  for 
making  research  accessible  to  practitioners  and  policymakers  is  the  What  Works 
Clearinghouse,  which  was  established  in  2002  by  the  Institute  of  Education  Sciences 
(http://www.whatworks.ed.gov/).  The  What  Works  Clearinghouse  provides  consumers  of 
education  research  with  user-friendly  reviews  of  the  effectiveness  of  replicable  education 
interventions  (programs,  products,  practices,  and  policies)  that  are  intended  to  improve 
student  outcomes. 

•  Encourage  state  and  local  partnerships  with  nearby  research  institutions  to  take  the  lead 
in  research  that  will  meet  the  needs  of  decision  makers  (Corcoran,  2003). 
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•  Make  it  a  program  requirement  for  grantees  to  develop  and  implement  a  dissemination 
plan  that  improves  access  to  and  use  of  research  findings  by  policymakers  and  decision 
makers  at  the  institutional,  local,  district,  state,  or  national  level  as  appropriate. 

•  Support  the  exchange  of  ideas  among  research,  policy,  and  practice. 

•  Develop  communication  and  dissemination  strategies  that  cut  across  projects  and 
programs. 


Closing  the  Research  to  Practice  Gap 

Past  efforts  to  link  the  domains  of  education  research  and  practice  have  often  been  disappointing. 
Practitioners  generally  do  not  read  the  journals  where  research  is  published  and  do  not  attend 
conferences  where  research  is  discussed.  Researchers  and  practitioners  often  do  not  share  the 
same  language.  In  addition,  practitioners  do  not  have  time  to  read  research  and  make  sense  of  it 
and  use  it  productively  in  the  classroom  (Even,  2003).  Conversely,  many  researchers  do  not 
spend  time  in  schools  and  therefore  don’t  fully  appreciate  the  complexities  of  educational 
settings. 

Barriers  That  Prevent  Teachers  From  Making  Use  of  Research 

Stipek  (2005)  points  out  that: 

•  most  research  evidence  is  published  in  places  and  forms  that  only  other  researchers  visit 
and  can  comprehend; 

•  basing  decisions  on  research  findings  is  not  part  of  the  culture  of  teaching;  and 

•  the  organization  of  teachers’  work  does  not  enable  them  to  acquire  a  deep  understanding 
of  innovative  methods  and  programs. 

Another  study  (Hemsley-Brown  &  Sharp,  2003)  has  also  identified  several  barriers  to  using 
research  in  education  including: 

•  The  large  (unruly)  volume  of  research  available; 

•  The  lack  of  specific  applicability  of  much  of  the  research;  and 

•  The  ambiguity  of  available  research  findings. 


What  can  be  done  to  overcome  these  barriers? 

For  research  to  be  accessible  and  useful  for  teachers  and  to  influence  teaching  and  learning, 
Judith  Sowder  (2000)  has  suggested  that  authors  of  research  papers  rewrite  their  research  reports 
for  a  more  general  audience  and  submit  them  to  journals  where  they  will  reach  practitioners. 

Burkhardt  and  Schoenfeld  (2003)  argue  that  research  could  be  more  useful  if  its  structure  and 
organization  were  better  linked  to  the  practical  needs  of  the  education  system.  They  suggest  that 
we  should  develop  research-based  tools  and  processes  for  practitioners  to  use  and  provide 
sustained,  long-tenn  professional  development  for  teachers.  They  also  suggest  that  universities 
should  create  academic  incentives  that  reward  researchers  for  their  contributions  to  practice. 
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Hemsley-Brown  and  Sharp  (2003)  note  that  in  their  studies,  teachers  are  more  likely  to  respond 
positively  to  quantitative  research  and  have  difficulty  making  use  of  qualitative  studies;  that 
strong  relationships  between  researchers  and  practitioners  are  necessary  as  is  a  professional 
culture  that  supports  research  application;  and  that  the  effect  of  research  on  schools  depends 
largely  upon  the  ability  of  teachers  to  understand,  critique,  and  directly  apply  the  findings. 

Active  collaborations  between  researchers  and  practitioners  to  identify  issues  of  primary 
importance  to  investigate  are  rare.  Instead  of  structuring  the  dissemination  activities  as  a  one¬ 
way  flow  of  infonnation,  more  sophisticated  dissemination  efforts  need  to  be  devised  and 
examined  that  could  involve  an  exchange  of  infonnation  and  ideas  between  researchers  and 
practitioners.  Design  research  is  one  promising  approach  to  increasing  collaboration  among 
teachers  and  researchers  (see  Education  Researcher,  Volume  32,  No.  1,  January/February  2003 
Theme  Issue:  The  Role  of  Design  in  Educational  Research).  One  of  the  reports  reviewed  by  the 
Task  Group  (National  Research  Council,  2003)  contains  a  number  of  recommendations  on  these 
issues.  The  study  concludes  that  “there  is  currently  no  institution  in  which  education  practitioners 
and  researchers  from  a  variety  of  disciplines  are  provided  with  support  to  interact,  collaborate, 
and  leam  from  each  other.  Thus,  researchers  often  fail  to  bring  important  understandings  to  the 
stage  of  usability,  and  practitioners  have  no  way  to  either  analyze  and  systematize  their  own 
wisdom  of  practice  or  to  influence  directions  and  shape  the  research  agenda.”  Hopefully,  the 
Science  of  Learning  Centers  funded  by  the  National  Science  Foundation  will  provide  one  major 
avenue  for  addressing  these  recommendations.  More  specifically,  these  Centers  are  being  built 
around  a  unifying  research  focus  that  will  incorporate  a  diverse,  multidisciplinary  environment 
involving  appropriate  partnerships  with  academia,  industry,  all  levels  of  education,  and  other 
public  and  private  entities. 

Finally,  it  should  be  noted  that  efforts  by  federal  agencies  to  better  integrate  education  research 
with  practice  are  frequently  constrained  by  the  administrative  and  established  organizational 
structures  in  which  reforms  must  be  implemented.  In  this  regard,  more  research  is  needed  on  the 
factors  that  are  critical  for  successfully  bringing  about  large-scale  organizational  change  within 
educational  systems  so  that  future  efforts  to  effect  needed  reforms  will  be  founded  on  a  solid 
evidence  base. 

How  do  our  federal  educational  research  programs  disseminate  research  findings  to 
practitioners? 

Results  from  the  program  solicitations  reviewed  here  indicate  that  the  three  federal  agencies 
inform  practitioners  about  the  research  they  fund  primarily  through  conferences  and  workshops, 
journal  articles,  newsletters,  and  agency  websites.  While  individual  projects  may  be  required  to 
work  directly  and  closely  with  practitioners,  rarefy  do  federal  programs  convey  the  results  of 
their  research  directly  to  practitioners. 

What  can  federal  educational  research  programs  do  to  better  integrate  research  with 
practice? 

There  are  a  number  of  ways  in  which  federal  agencies  can  help  to  better  integrate  education 
research  and  practice,  including: 
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•  Encourage  federally-funded  research  projects  to  communicate  their  findings  to  multiple 
audiences  (researchers,  practitioners,  policymakers). 

•  Encourage  the  use  of  different  modes  of  communicating  research  findings  (face-to-face, 
web-based,  publication  in  practitioner  venues). 

•  Engage  practitioners  to  collaborate  with  researchers  in  setting  research  agendas. 

•  Focus  research  on  the  core  problems  of  practice. 

•  Seek  well-designed  projects  that  engage  teachers  in  the  research  process. 

•  Engage  practitioners  along  with  researchers  as  peer  reviewers  in  every  milestone  of  the 
research  project’s  lifespan,  identifying  when  research-based  knowledge  is  good  enough 
to  inform  practice  and  policy. 

•  Provide  support  through  an  established  institution,  such  as  professional  development 
schools,  to  enable  practitioners  and  researchers  to  interact,  collaborate,  and  learn  from 
each  other. 

•  Fund  syntheses  of  data  across  studies  and  link  it  to  school  practice  in  a  wide  variety  of 
school  settings. 

•  Bridge  theory  and  practice  by  conducting  “use-inspired”  research  focused  on  improving 
classroom  learning  and  teaching. 

•  Encourage  connections  between  practitioners,  researchers,  and  technical  assistance 
providers. 

•  Support  projects  that  create  lasting  bonds  between  education  and  science  communities, 
moving  beyond  research  and  education  as  side-by-side  activities. 

•  Support  long-term  collaborative  work,  moving  away  from  episodes  of  cooperation. 

•  Support  projects  that  establish  horizontal  connections  between  and  among  disciplinary 
scientists,  scientists  studying  learning  and  teaching,  and  education  researchers  and 
evaluators. 
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Section  IV:  Summary  of  Recommendations  Drawn  From  STEM  Education 
Research,  Policy,  and  Workforce  Reports 

Appendix  E  contains  a  list  of  the  STEM  education  and  workforce  reports  that  were  reviewed  in 
an  effort  to  carefully  examine  the  kinds  of  issues  and  concerns  raised  about  STEM  education 
research  from  1995  to  2006.  Although  this  list  cannot  be  considered  exhaustive,  it  is  certainly 
representative  of  the  range  of  STEM  education  and  workforce  reports  published  over  the  past 
decade.  It  includes  numerous  reports  issued  by  the  National  Academy  of  Sciences,  as  well  as  by 
the  RAND  Corporation,  the  Carnegie  Foundation,  the  Educational  Testing  Service,  the  National 
Science  Board,  and  the  White  House  Office  of  Science  and  Technology  Policy,  among  others.  A 
total  of  64  reports  were  consulted,  from  which  literally  hundreds  of  STEM  education-related 
issues,  ideas,  concerns,  suggestions,  and  recommendations  were  extracted.  However,  the  vast 
majority  of  these  were  not  directed  toward  education  research ;  and  since  this  was  the  focus  of 
the  Task  Group’s  charge,  only  recommendations  pertaining  to  STEM  education  research  were 
considered  in  preparing  the  present  report.  This  list  of  recommendations  is  provided  in 
Appendices  F  and  G,  broken  down  into  two  main  categories — recommendations  about  research 
and  recommendations  about  processes  related  to  research.  From  left  to  right,  the  columns  in 
Appendix  F  include  the  Report  #,  the  relevant  STEM  area  (science  technology,  engineering 
math)  or  a  cross-cutting  domain  (design/method/measurement,  evaluation,  or  miscellaneous),  the 
program  priority  area  drawn  from  Table  3,  if  appropriate  (for  example,  “capacity  building”  or 
“evaluation  &  assessment”),  followed  by  the  specific  recommendation(s)  from  the  listed  report, 
and  finally  any  programs  from  the  federal  agencies  that  address  the  recommendation  (as  derived 
from  Table  3  and  Appendices  C  and  D). 

Research  Recommendations 

The  topics  in  need  of  additional  research,  as  identified  by  these  reports,  can  be  clustered  into 
seven  primary  groups: 

•  Research  on  teachers,  including  pedagogy,  content  knowledge,  career  patterns,  etc. 

•  Research  on  student  learning 

•  Research  on  assessment  and  how  to  design  effective  assessments  in  STEM  education 
fields 

•  Evaluation  of  interventions 

•  Research  on  the  education  of  women  and  underrepresented  groups 

•  Research  on  career  patterns  of  undergraduate,  graduate,  and  career  scientists 

•  Communication  and  synthesis  of  research  results 


Process  Recommendations 

The  process  recommendations  drawn  from  the  various  STEM  education  reports  are  listed  in 
Appendix  G.  These  recommendations  can  be  clustered  as  follows: 

•  Synthesis  of  available  knowledge 

•  Development  of  national  databases 
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•  Better  links  among  research,  policy,  and  practice 

•  Evaluation  of  programs 

•  Employment  in  research  of  a  wide  array  of  both  quantitative  and  qualitative  methods 

•  Accumulation  of  knowledge 

•  Coordination  and  cooperation  among  federal  agencies 

•  The  use  of  rigorous  methods  and  designs  in  education  research 

•  Increased  federal  investment  in  research 

•  Focus  and  selectivity  in  research  funding 

•  Support  for  consortia  and  multi-institution  programs 

•  Focus  on  the  role  of  technology  both  as  an  aid  to  research  and  as  a  subject  of  research 
study 


Additional  Research  and  Process  Recommendations  Drawn  From  the  U.S.  Department  of 
Education’s  2003  Mathematics  and  Science  Initiative 

The  Mathematics  and  Science  Initiative  (MSI)  was  launched  by  the  former  Secretary  of 
Education,  Dr.  Rod  Paige,  in  February  2003  with  three  main  goals:  1)  to  engage  the  public  in 
recognizing  the  need  for  better  mathematics  and  science  education  for  all  children;  2)  to  initiate  a 
campaign  to  recruit,  prepare,  train,  and  retain  teachers  with  strong  backgrounds  in  mathematics 
and  science;  and  3)  to  develop  a  research  base  to  improve  our  knowledge  of  what  boosts  student 
learning  in  mathematics  and  science.  The  aim  of  the  research  portion  of  the  MSI  was  to  establish 
a  foundation  of  evidence  upon  which  efforts  to  improve  student  achievement  levels  in 
mathematics  and  the  sciences  can  be  based.  Although  this  initiative  is  no  longer  in  operation,1 
several  of  the  research  domains  that  were  identified  as  essential  for  achieving  its  objectives  are 
still  relevant  to  the  issues  being  examined  in  the  present  report,  and  as  such  are  included  in  this 
section: 

Cognitive  foundations  of  mathematics  and  science  learning 

Basic  knowledge  of  how  people  acquire,  process,  and  apply  scientific  and  mathematical 
knowledge  is  fundamental  to  the  development  of  effective  educational  practice.  Research  that 
produces  scientifically  credible  findings  about  student  cognition,  motivation,  and  development  in 
mathematics  and  sciences  can  provide  a  foundation  of  knowledge  to  inform  educational  practice 
and  would  fulfill  part  of  the  mandate  of  the  NCLB  Act.  Research  in  this  area  should  focus  on 
identifying  the  cognitive  and  motivational  processes  that  undergird  the  acquisition  and 
maintenance  of  proficiency  in  mathematics  and  the  sciences.  (Federal  research  programs,  such  as 
NSF's  programs  in  Developmental  and  Feaming  Sciences  and  Cognitive  Neuroscience  and  ED's 
Cognition  and  Student  Fearning  program,  and  NICHD’s  Mathematics  and  Science  Cognition  and 
Feaming  Program  support  research  in  this  area.) 


1  The  Department  of  Education  under  the  leadership  of  the  current  Secretary,  Margaret  Spellings,  has  not  only 
continued  to  make  math  and  science  education  a  top  priority,  but  has  also  underscored  the  importance  of 
scientifically  based  research  for  advancing  the  teaching  and  learning  of  these  subjects  (see  for  example  a  description 
of  the  National  Mathematics  Advisory  Panel,  http://www.ed.gov/about/bdscomm/list/mathpanel/factsheet.html, 
formed  by  the  Secretary  in  response  to  an  Executive  Order  issued  by  President  Bush  on  April  18,  2006: 
http://www.whitehouse.gov/news/releases/2006/04/2006Q418-5.html). 
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Identify  and  develop  effective  interventions  in  mathematics  and  science  education 
Research  that  investigates  specific  teaching  methods  and  curriculum  materials  will  help  identify 
the  most  effective  instructional  approaches.  Effective  instruction  requires  teaching  methods  and 
instructional  materials  that  are  appropriate  to  the  ability  and  maturity  of  the  students.  Work  in 
this  area  should  identify  the  instructional  conditions  under  which  students  from  varying  abilities 
and  backgrounds  leam  mathematics  and  science.  Based  on  available  evidence,  key  areas  crucial 
for  supporting  mathematics  and  science  education  include  approaches  to  instruction  and 
curricular  content  and  fonnat.  (NSF  has  several  long-standing  programs  that  provide  the  R  and  D 
for  teacher  professional  development,  assessment  of  student  learning,  and  development  of 
materials  and  curricula.  A  primary  goal  of  ED’s  research  agenda  is  to  provide  schools  with 
evidence  of  the  effectiveness  of  educational  interventions.) 

Identify  effective  models  for  training  mathematics  and  science  teachers 
Research  in  this  area  should  examine  the  effectiveness  of  different  models  of  selection,  training, 
and  professional  development  of  mathematics  and  science  teachers.  Appropriate  targets  for 
research  include  the  effects  of  different  routes  of  entry  into  teaching,  the  different  skills  and 
abilities  that  are  required  to  teach  mathematics  and  science  at  different  levels  and  for  different 
types  of  students;  the  form  and  duration  of  pre-professional  coursework  that  is  optimal  for 
different  types  of  teaching;  the  role  of  induction  experiences,  field  work,  and  ongoing 
professional  development  in  developing  effective  teachers;  the  effects  of  differentiated  staffing 
on  the  effectiveness  of  instruction  at  different  levels  of  K-12  education;  and  mechanisms  for 
teacher  recruitment  and  retention.  (This  research  area  is  central  to  the  missions  of  both  NSF  and 
the  Department  of  Education.) 

Develop  and  evaluate  technologies  that  can  advance  and  extend  student  learning 
Mathematics  and  science  learning  are  areas  in  which  learning  applications  that  allow  students  to 
go  beyond  the  restrictions  of  their  classroom  and  teacher  need  to  be  expanded  and  evaluated. 
Because  many  areas  of  mathematics  and  science  learning  require  students  to  be  engaged  in  ways 
that  are  difficult  to  arrange  in  traditional  classroom  instruction,  this  area  of  work  should  focus  on 
ways  to  deliver  individualized  instruction  that  is  sensitive  to  student’s  abilities,  levels,  and 
approaches  to  learning.  In  addition,  the  impact  of  innovations  such  as  on-line  homework  and 
distance  education  need  to  be  evaluated  more  thoroughly.  (The  Institute  of  Education  Sciences 
supports  work  in  this  area  through  a  number  of  its  research  programs.) 

Develop  reliable  and  valid  assessments  of  mathematics  and  science  learning 
Carefully  developed  assessment  tools  are  required  to  judge  the  progress  of  students,  schools,  and 
the  nation,  in  achieving  higher  levels  of  proficiency  in  mathematics  and  science.  Building  on 
knowledge  of  the  foundations  of  mathematical  and  scientific  competence,  research  on 
assessment  should  develop  and  evaluate  the  technical  adequacy  (i.e.,  psychometric  properties) 
and  practical  utility  (e.g.,  instructional  applications)  of  tests  designed  to  assess  proficiency  levels 
in  mathematics  and  science  education.  (NSF  and  ED  are  supporting  research  that  addresses  this 
need.) 

Understand  how  to  organize  schools  and  design  instructional  policies 

Work  in  this  area  should  examine  how  the  organization  of  schools  in  the  form  of  instructional 

leadership,  staff  involvement,  school  and  class  size,  scheduling  of  opportunities  for  learning; 
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parental  and  community  support;  and  accountability  systems  within  schools  affect  student 
outcomes.  Research  should  also  investigate  the  effects  of  different  district-  and  state-level 
policies  such  as  alignment  of  standards  and  accountability  systems  and  different  forms  of 
performance  compensation.  (ED  is  supporting  research  to  address  this  need  through  the  Institute 
of  Education  Sciences  research  program  on  Education  Policy,  Finance,  and  Systems.) 

Understand  student  disabilities  that  hinder  mathematics  and  science  learning 
Learning  disabilities  now  account  for  more  than  half  of  all  students  enrolled  in  special  education. 
NICHD  supports  research  that  explores  the  cognitive,  perceptual,  behavioral,  genetic,  honnonal, 
and  neurobiological  mechanisms  that  are  influential  in  the  expression  of  mathematical  learning 
abilities  and  learning  disabilities,  predictors  of  disabilities,  and  the  development  of  preventive 
and  treatment  approaches  to  ameliorate  mathematics-related  learning  disabilities.  ED  supports 
research  to  evaluate  the  effectiveness  of  mathematics  and  science  curricula  and  instructional 
practices  for  improving  student  learning  and  achievement  for  students  with  identified  disabilities 
and  students  at  risk  for  disability  through  the  Mathematics  and  Science  Special  Education 
Research  Program. 

Identify  the  competencies  essential  for  a  workforce  well  trained  in  mathematics  and  the 
sciences 

Much  mathematics  and  science  education  is  based  on  ideas  about  what  students  need  to  know 
that  are  drawn  from  research  and  professional  consensus.  These  approaches  are  then 
incorporated  in  standards  documents,  such  as  those  created  by  the  National  Council  of  Teachers 
of  Mathematics,  the  AAAS  Benchmarks  and  the  National  Science  Education  Standards  from  the 
National  Research  Council.  Another  route  to  setting  standards  and  expectations  is  based  on  an 
analysis  of  the  competencies  that  are  required  to  perform  mathematical  and  scientific  tasks  as 
they  are  encountered  in  the  world  of  work.  The  goal  here  is  to  identify  the  areas  of  mathematical 
and  scientific  knowledge  required  for  professional  competence  in  a  variety  of  areas,  as  well  as 
the  specific  content  knowledge  and  skills  needed  to  work  in  professions  designated  as  “high- 
need.”  In  addition,  research  in  this  area  should  examine  equity  of  educational  access  and 
investigate  ways  of  improving  the  diversity  of  the  workforce  and  professions  that  rely  on 
mathematical  and  scientific  skills. 

Furthermore,  research  will  help  determine  the  kinds  of  mathematics  and  science  skills  needed  by 
individuals  who,  after  being  in  the  workforce,  make  career  changes  that  require  more  detailed 
knowledge  of  these  domains. 

A  comprehensive  research  program  on  mathematics  and  science  learning  will  require  support 
from  foundations,  universities,  and  the  private  sector  in  addition  to  federal  agencies.  One  goal  of 
such  a  program  is  to  develop  strategies  for  more  effective  collaborative  efforts  and  infonnation 
sharing  across  these  entities. 
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Section  V:  Federal  Agency  STEM  Education 
Research  Portfolios:  How  Well  are  We  Addressing  the  Recommendations? 

Discussion  of  Research  Recommendations 

The  Task  Group’s  examination  of  the  distribution  of  program  objectives  in  Table  3  and  the 
supporting  statements  in  Appendices  C  and  D  indicates  that  the  federal  agencies  are  currently 
addressing  most  of  the  research  recommendations  summarized  in  Section  IV,  at  least  to  some 
extent.  Each  of  the  research  recommendations  will  be  discussed  below  including  areas  where 
gaps  remain  either  in  the  kind  of  work  currently  being  supported  or  in  the  process  of  being 
solicited.  In  addition,  where  applicable,  comments  about  the  efficiency  of  program  objectives 
within  and  across  agencies  will  be  noted,  focusing  on  the  degree  of  overlap  and  whether  better 
coordination  is  recommended  to  maximize  the  impact  of  our  joint  investments. 

•  Teachers 


This  category  includes  research  pertaining  to  the  identification  of  effective  teacher 
licensure  tests;  dynamics  of  teacher  performance  and  effectiveness;  what  practices 
enable  teachers  to  help  students  develop  mathematical  proficiency;  developing 
teachers’  knowledge  of  mathematics  and  science;  and  the  best  ways  to  educate, 
train  and  evaluate  teachers  in  21st  century  skills. 

Examining  the  number  of  federal  agency  programs  that  promote  priorities  relating 
to  “Teacher  Development”  and  “Curriculum  or  Instructional  Practice,”  reveals  that 
a  majority  (56  percent)  of  the  32  agency  programs  represented  in  Table  3  focus  on 
the  needs  of  teachers.  For  example,  one  stated  priority  of  the  ED-02  program  is  “To 
identify  effective  strategies  for  improving  the  performance  of  classroom  teachers  in 
ways  that  increase  student  learning  and  school  achievement.”  Similarly,  one  of  the 
NSF-10  program  objectives  is  “To  synthesize  and  further  advance  a  compelling 
body  of  research  that  will  both  infonn  and  strengthen  STEM  teacher  effectiveness 
and  classroom  instruction.” 

Gaps :  The  Task  Group  did  not  identify  any  major  programmatic  gaps  in  this  area. 
However,  consistent  with  current  efforts  of  the  federal  agencies  to  strengthen  the 
research  base  in  teacher  education,  the  reader  may  wish  to  consult  a  report  released 
in  2005  by  the  American  Educational  Research  Association 
(http://www.aera.net/publications/?id=793).  Entitled  Studying  Teacher  Education, 
the  report  provides  a  comprehensive  review  and  analysis  of  extant  research  in  this 
area  and  also  proposes  a  research  agenda.  Of  relevance  to  STEM  fields,  it  was 
found  that  certification  in  math  education  is  one  indicator  of  effective  teaching  and 
successful  student  achievement  in  math.  Moreover,  the  data  reveal  that  prospective 
teachers’  study  of  mathematics  in  college  is  correlated  with  the  mathematics 
learning  of  their  high  school  students. 

Efficiency.  On  the  one  hand,  the  rather  extensive  focus  on  teacher  development 
across  at  least  half  of  the  agency  programs  examined  here  is  consistent  with  the 
recommendations  coming  from  many  quarters  regarding  an  urgent  need  to  improve 
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our  understanding  in  this  area.  On  the  other  hand,  it  is  not  yet  clear  whether  greater 
coordination  is  needed  to  ensure  that  these  efforts  are  complementary  rather  than 
duplicative.  A  more  detailed  analysis  is  needed  to  properly  assess  this  issue;  the 
evaluation  should  also  ascertain  the  degree  of  overlap  among  the  kinds  of 
applications  submitted  in  response  to  the  solicitations. 

•  Student  learning 


This  cluster  includes  research  recommendations  about  identifying  developmental 
models  of  cognition  and  learning;  learning  mathematics  practices  that  are  essential 
to  effective  day-to-day  use  of  mathematics;  large-scale  studies  to  investigate  the 
impact  of  standards-based  curricula  on  student  achievement;  and  consensus  on  a 
common  core  of  mathematics  and  science  knowledge  and  skills. 

As  can  be  observed  from  Table  3,  50  percent  of  the  agency  programs  encourage 
research  in  “Student  Cognition  &  Learning.”  For  example,  one  of  the  objectives  of 
the  ED-02  program  is  “To  improve  student  learning  by  bringing  recent  advances  in 
cognitive  science  to  bear  on  significant  problems  in  education.”  In  addition,  the 
NUT-02  program  lists  as  one  its  objectives  “To  improve  our  understanding  of  the 
cognitive  and  developmental  bases  of  scientific  thinking  and  learning.” 

Gaps :  Although  the  Task  Group  did  not  identify  any  major  gaps  in  this  area,  a 
recently  released  pre-publication  version  of  a  forthcoming  report  from  the  National 
Academy  suggests  that  to  the  contrary,  some  important  gaps  do  exist  -  at  least  with 
respect  to  research  and  development  in  science  education.  The  report,  entitled 
Taking  Science  to  School:  Learning  and  Teaching  Science  in  Grades  K-8,  was  co¬ 
sponsored  by  NSF,  NICFID/NIH,  and  the  Merck  Institute  for  Science  Education 
(http://fermat.nap.edu/books/0309102057/html).  Among  other  significant 
contributions,  the  study  committee  developed  a  framework  and  definition  for  what 
it  means  to  be  proficient  in  science,  consisting  of  four  intertwined  and  equally 
important  strands:  a)  students  should  know,  use,  and  interpret  scientific 
explanations  of  the  natural  world;  b)  they  should  be  able  to  generate  and  evaluate 
scientific  evidence  and  explanations;  c)  they  should  understand  the  nature  and 
development  of  scientific  knowledge;  and  d)  they  should  participate  productively  in 
scientific  practices  and  discourse.  Of  particular  relevance  to  the  present  analysis  are 
the  recommendations  the  study  committee  provides  with  respect  to  policy  and 
practice.  And  possibly  the  most  important  conclusion  they  arrive  at  in  this  regard  is 
that  there  is  a  conspicuous  “lack  of  an  infrastructure  for  research,  development,  and 
implementation  in  science  education  that  is  informed  by  research  on  fundamental 
aspects  of  learning  and  teaching  but  takes  up  problems  and  questions  that  are 
grounded  in  realities  of  practice.”  They  go  on  to  offer  specific  suggestions  for  ways 
to  improve  this  state  of  affairs. 

It  is  also  worth  noting  here  that  within  less  than  a  week  following  the  release  of  the 
pre-publication  version  of  this  National  Academy  report,  the  National  Science 
Teachers  Association  (NSTA)  posted  a  “discussion  draft”  on  their  website  building 
directly  one  of  the  key  recommendations  emerging  from  this  report.  More 


36 


specifically,  the  NSTA’s  “Core  Ideas  (Science  Anchors)  in  Science  Education” 
begins  by  quoting  the  report  as  follows:  “The  next  generation  of  science  standards 
and  curricula  at  the  national  and  state  levels  should  be  centered  on  a  few  core  ideas 
and  should  expand  on  them  each  year,  at  increasing  levels  of  complexity,  across 
grades  K-8.  Today's  standards  are  still  too  broad,  resulting  in  superficial  coverage 
of  science  that  fails  to  link  concepts  or  develop  them  over  successive  grades.”  The 
draft  then  goes  on  to  note  that  “These  core  ideas  would  provide  an  “anchor”  and  a 
national  coherence  with  what  we  can  expect  all  students  to  learn  .  .  .  Science 
Anchors  will  help  teachers  to  better  manage  their  instruction  and  will  also  help 
stakeholders,  such  as  professional  societies,  textbook  companies,  professional 
development  providers,  assessment  providers,  and  others  to  work  from  a  core  set  of 
agreed-upon  key  ideas  in  the  sciences  that  are  clearly  aligned  with  assessments 
when  developing  ancillary  and  support  materials  for  the  science  education  market.” 

Efficiency.  The  Task  Group  has  been  able  to  determine  that  there  is  comparatively 
little  overlap  in  the  majority  of  the  kinds  of  research  being  supported  in  this  area  by 
the  ED  and  NIH.  For  example,  the  large  majority  of  NIH  studies  focus  on  more 
laboratory-based,  cognitive  research  in  mathematical  thinking  and  scientific 
reasoning,  while  the  objectives  of  ED’s  IES  solicitations  tend  to  focus  on  studies  in 
the  context  of  everyday  instructions  settings,  building  upon  the  findings  of  the  NIH 
type  of  research.  Indeed,  this  is  stated  explicitly  in  the  description  of  the  IES 
Cognition  and  Student  Learning  Program:  “The  purpose  of  the  Cognition  and 
Student  Learning  Education  research  program  is  to  improve  student  learning  by 
bringing  recent  advances  in  cognitive  science  to  bear  on  significant  problems  in 
education.”  There  appears  to  be  a  somewhat  greater,  albeit  limited  degree  of 
overlap  between  portions  of  the  NSF  ROLE  (Research  on  Learning  in  Education) 
Program  and  the  NIH/NICHD  Mathematics  and  Science  Cognition  and  Learning 
portfolios.  These  programs  should  be  carefully  coordinated  to  capitalize  on 
opportunities  for  collaboration,  maximize  the  impact  of  the  investments,  and  avoid 
duplication  of  effort. 

•  Assessment  and  evaluation  of  interventions 


Included  in  this  category  is  research  focused  on  the  design  of  assessments  that  yield 
valid  and  fair  inferences  about  student  achievement  or  instructional  interventions. 

In  Table  3,  43  percent  of  the  agency  programs  have  components  that  specify 
“evaluation  and  assessment”  research.  For  example,  one  of  the  objectives  under 
NSF-05  is  “To  develop  new  assessment  materials  (tools)  and  processes  for  use  in 
single  or  multiple  undergraduate  disciplines.”  Additionally,  the  NSF-02  program 
includes  a  priority  “To  develop  and  document  the  psychometric  properties  of  test 
items  that  are  designed  to  measure  learning  critical  to  scaling  up  research,  e.g.,  to 
develop  measures  that  assess  the  fidelity  of  implementations,  student  knowledge, 
teacher  knowledge,  or  other  important  predictor  or  outcome  variables  related  to 
scale-up.  The  study  of  measures  that  use  technology  as  an  essential  component  is 
especially  encouraged,  e.g.,  a  variety  of  concept  inventories  are  currently  available 
in  several  science  disciplines;  studies  of  their  reliability  and  validity  are  important 
before  they  can  be  used  widely  for  scaling  up  research.” 
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With  respect  to  evaluation  of  interventions,  a  major  priority  of  the  ED-08  program 
is  “To  provide  federal  support  for  evaluations  of  the  effectiveness  of  education 
interventions  that  are  being  used  in  the  field,  that  appear  promising  based  on  student 
performance  or  fill  an  unmet  need,  but  that  have  not  benefited  from  a  rigorous 
evaluation  of  effectiveness.” 

Gaps :  Despite  the  emphasis  on  assessment  in  the  programs  cited  above,  it  is  clear 
that  there  remain  significant  gaps  in  our  knowledge  about  assessment,  especially  in 
the  area  of  student  achievement.  To  make  some  headway  in  this  regard,  two  recent 
efforts  are  discussed  here: 

1)  The  U.S.  Department  of  Education’s  former  Mathematics  and  Science  Initiative 
(MSI)  working  group,  composed  of  several  federal  agencies  as  well  as  relevant 
professional  associations  and  other  interested  parties,  organized  a  series  of 
conferences  on  mathematics  and  science  to  discuss  current  issues  in  these  areas, 
including  assessment.  In  addition,  in  January  of  2004,  the  U.S.  Department  of 
Education  (ED),  in  cooperation  with  the  Council  of  Chief  State  School  Officers 
(CCSSO),  held  a  meeting  on  the  topic  of  state  science  assessments.  The  purpose 
was  to  discuss  issues  related  to  the  state  science  assessments  required  by  2007-2008 
under  the  No  Child  Left  Behind  act  and  how  both  organizations  can  be  of  help  as 
states  develop  and  implement  high  quality  science  assessments.  Attendees 
represented  the  following  groups:  1)  state  assessment  directors;  2)  testing 
companies,  including  those  that  may  be  subsidiaries  of  or  affiliated  with  curricula 
and  textbook  publishers;  3)  cognitive  researchers  and  psychometricians;  and  4) 
scientists  and  engineers.  The  group  concluded  that  too  few  high  quality  science 
assessments  exist  and  that  developing  a  “tool  kit”  for  those  charged  with  producing 
them  should  be  a  priority.  Representatives  from  a  number  of  Federal  agencies  and 
other  organizations  attended  as  observers.  (Meeting  proceedings  can  be  found  at 
http://www.ed.gov/rschstat/research/progs/mathscience/science- 

assessment/proceedings  .doc) . 

2)  NSF  has  long  supported  research  and  development  projects  in  the  use  of  new 
tools  and  technologies  for  creating  assessments  that  provide  more  complete 
information  about  students’  STEM  understanding.  Furthermore,  the  National 
Research  Council  has  just  released  a  report,  Systems  for  State  Science  Assessment 
(2005)  following  two  years  of  study.  The  report  provides  advice  to  states  on 
designing  and  developing  coherent  science  assessment  systems  in  light  of 
requirements  of  the  No  Child  Left  Behind  Legislation  (NCLB).  Specific 
components  of  the  report  include:  a)  a  conceptual  framework  based  on  the  report 
Knowing  What  Students  Know  and  other  recent  findings  on  assessing  students’ 
understanding  of  important  science  ideas;  b)  four  models  and  methods  for 
developing  high  quality  science  assessments  based  on  this  conceptual  framework, 
created  by  expert  design  teams  that  result  in  interesting  and  rich  choices  that  states 
can  adapt  or  adopt;  and  c)  a  collaborative  process  that  included  key  stakeholders  in 
states  and  school  districts  so  that  the  guidance  provided  is  practical  and  usable. 
Science  assessment  experts,  scientists  and  science  educators,  policymakers,  test 
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developers  and  others  concerned  about  science  have  had  few  opportunities  to 
discuss  issues  in  science  assessment  together. 

Efficiency.  As  with  the  other  areas  mentioned  above,  further  analysis  of 
opportunities  for  collaboration  between  programs  with  similar  objectives,  both 
within  and  across  agencies,  is  called  for. 

•  Education  of  women  and  underrepresented  groups 

Examination  of  Table  3  for  programs  with  research  objectives  relating  to  diversity 
and  broadening  participation  in  STEM  fields  reveals  that  16  percent  of  the  32 
programs  are  concerned,  at  least  in  part,  with  the  impact  of  race/ethnicity,  19 
percent  with  the  impact  of  socioeconomic  status  (SES),  9  percent  with  the  role  of 
gender,  12  percent  with  broad  disabilities,  and  6  percent  specifically  with  learning 
difficulties.  With  regard  to  disabilities,  for  example,  an  objective  of  the  NSF-07 
program  is  to  “increase  the  participation  and  achievement  of  persons  with 
disabilities  in  science,  technology,  engineering,  and  mathematics  (STEM)  education 
and  careers.”  The  NIH-1  program  includes  a  major  focus  on  learning  difficulties  in 
math,  with  the  objective  of  delineating  “the  nature  and  extent  of  specific  learning 
disabilities  in  mathematics,  including  diagnosis,  classification,  etiology,  prevention, 
and  treatment.”  With  respect  to  race/ethnicity,  an  objective  of  the  ED-06  program  is 
“To  support  the  identification  of  interventions  and  approaches  in  mathematics 
education  that  will  result  in  closing  achievement  gaps  between  minority  and  non¬ 
minority  students.”  SES  is  emphasized  in  several  programs.  For  example,  in  the 
ED-07  program,  one  goal  is  “To  understand  the  impact  of  school  improvement 
programs  on  instruction  and  student  perfonnance  in  language  arts  and  mathematics 
in  high-poverty  elementary  schools.” 

With  respect  to  gender,  a  priority  of  the  one  of  the  NSF  programs,  NSF-06,  is  “To 
broaden  the  participation  of  girls  and  women  in  STEM  education  by  supporting 
research,  dissemination  of  research,  and  integration  of  proven  good  practices  that 
will  ultimately  lead  to  a  larger  and  more  diverse  domestic  science  and  engineering 
workforce.” 

Gaps :  Although  various  programs  at  ED,  NSF,  and  NIH  target  different 
components  of  equity  as  described  briefly  above,  it  is  clear  that  more  could  be  done 
in  some  of  these  domains.  With  respect  to  race/ethnicity  and  SES,  for  example, 
NSF's  Urban  Systemic  Program  (USP)  has  supported  several  research  and 
evaluative  studies  on  the  most  promising  approaches  for  helping  students  in  large 
urban  districts  succeed  in  STEM  (See  Kim  et  ah,  Academic  Excellence  for  all 
Urban  Students,  2001  and  Fortus  et  ah,  Design-Based  Science  and  Student 
Learning,  Journal  of  Research  in  Science  Teaching,  Vol.  41,  Issue  10,  December 
2004).  Additional  initiatives  of  this  type  are  certainly  needed.  As  noted  earlier  this 
report,  ED’s  Institute  of  Education  Sciences  has  recently  launched  its  new  National 
Center  for  Special  Education  Research  (NCSER).  Through  the  Mathematics  and 
Science  Special  Education  Research  Program  within  this  Center,  IES  will  support 
research  to  evaluate  the  effectiveness  of  mathematics  and  science  curricula  and 
instructional  practices  for  improving  learning  and  achievement  for  students  with 
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identified  disabilities  and  students  at  risk  for  disability.  While  this  constitutes  a 
major  step  in  the  right  direction  for  dealing  with  disabilities,  all  federal  agencies 
that  fund  disability-related  STEM  education  research  could  no  doubt  place  an  even 
greater  emphasis  on  this  topic. 

With  respect  to  the  role  of  gender  in  math  and  science  learning,  as  well  as  in  career 
choice  and  workforce  participation,  much  remains  to  be  done.  For  example,  despite 
some  intriguing  findings  about  contributing  factors,  large  gaps  remain  in  our 
knowledge  regarding  the  reasons  why  women  remain  underrepresented  in  STEM 
fields.  For  the  health  of  our  future  workforce,  it  is  imperative  that  federal  agencies  it 
a  priority  to  resolve  these  issues.  The  success  of  future  strategies  will  be  enhanced 
to  the  extent  that  they  are  informed  by  rigorous  psychological,  sociological, 
educational,  and  economic  research.  Toward  this  end,  the  National  Research 
Council’s  Committee  on  Science,  Engineering  and  Public  Policy  (COSEPUP) 
recently  released  a  consensus  report,  entitled  “Beyond  Bias:  Fulfilling  the  Potential 
of  Women  in  Academic  Science  and  Engineering. 
http://newton.nan.cdu/cataloCl  1741.html.” 

•  Career  patterns  of  undergraduate,  graduate  and  career  scientists 

Observation  of  the  “STEM  Workforce  Improvement”  column  in  Table  3  illustrates 
that  12%  of  the  federal  agency  programs  focus  on  projects  that  relate  to  career 
patterns. 

Gaps :  To  the  extent  that  their  authority  permits,  the  agencies  should  consider 
placing  higher  priority  on  this  area  than  is  currently  the  case.  The  rapidly  growing 
demand  for  workers  with  STEM  knowledge  highlights  the  urgent  need  for  gaining  a 
better  understanding  of  the  preparedness  and  career  choices  of  young  people 
entering  the  workforce.  Of  the  major  agencies  examined  in  this  report,  only  the 
National  Science  Foundation  is  supporting  research  that  specifically  pertains  to 
these  issues.  We  suggest  that  a  federal  interagency  effort  may  be  needed  to 
efficiently  explore  linkages  between  STEM  workforce  research  and  education 
research  in  curriculum  and  instructional  practices,  equity,  and  student  cognition  and 
learning.  A  collaborative  approach  of  this  type  would  leverage  existing  resources  in 
a  manner  that  could  enhance  our  understanding  of  the  factors  that  give  rise  to  this 
national  problem  and  provide  evidence-based  corrective  actions  for  effectively 
modifying  the  current  imbalance  between  STEM  workforce  supply  and  demand. 

•  Communication  and  synthesis  of  research  results 

To  some  extent  the  federal  programs  in  Table  3  concerned  with  Schoolwide  and 
Systemwide  Reforms  and/or  STEM  Education  Policies  are  aimed  in  part  at  issues 
pertaining  to  communication  of  research  results.  For  example,  a  priority  of  the 
NSF-08  program  is  “To  capitalize  on  important  developments  across  a  wide  range 
of  fields  related  to  human  learning  and  to  STEM  education  with  research  across  a 
continuum  that  includes  the  diffusion  of  STEM  innovations.”  In  addition,  the  NSF- 
02  program  states  that  one  of  its  objectives  is  “To  provide  State  and  local  policy 
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makers,  as  well  as  school-level  administrators  and  university  faculty  and 
administrators,  with  information  on  efforts  at  improvement  that  have  led  to 
increased  and  sustained  student  learning.” 

Gaps :  Despite  increased  efforts  in  this  area,  the  members  of  the  Task  Group  agree 
that  there  is  much  to  be  done  before  research  results  are  adequately  synthesized  in  a 
manner  that  will  permit  effective  communication  to  both  policymakers  and 
practitioners,  among  other  audiences. 

Process  Recommendations 

The  process  recommendations  summarized  in  Section  IV  can  be  categorized  into  two  groups: 
The  first  group  consists  of  areas  in  which  the  Task  Group  found  agencies  to  be  making 
satisfactory  progress;  the  second  group  consists  of  domains  in  which  additional  attention  is 
recommended. 

Process  recommendations  that  the  federal  agencies  are  addressing  well 

•  Evaluation  of  programs 

In  October  of  2005,  the  Government  Accountability  Office  (GAO)  released  a 
report  entitled,  “Higher  Education:  Federal  Science,  Technology,  Engineering, 
and  Mathematics  Programs  and  Related  Trends.”  Among  other  objectives,  a 
major  aim  of  this  study  was  to  detennine:  a)  the  number  of  federal  civilian 
education  programs  funded  in  FY  2004  which  were  designed  specifically  to 
increase  the  number  of  students  and  graduates  pursuing  STEM  degrees  and 
occupations,  or  improve  educational  programs  in  STEM  fields,  and  b)  what 
agencies  report  about  the  effectiveness  of  these  programs.  According  to  this 
study,  out  of  207  STEM  education  programs  in  13  federal  civilian  agencies, 
officials  reported  that  evaluations  were  either  completed  or  in  progress  for  only 
about  half  of  them.  After  reviewing  these  and  other  findings,  the  GAO  concluded 
that  “little  is  known  about  the  extent  to  which  most  STEM  programs  are 
achieving  their  desired  results”  (GAO-06-1 14,  2005,  p.  18).2 

In  contrast  to  the  GAO’s  general  conclusion,  the  STEM  education  research 
programs  evaluated  for  this  report  have  demonstrated  a  growing  emphasis  on 
evaluating  effectiveness.  For  instance,  a  major  goal  of  the  Institute  of  Education 
Sciences’  Teacher  Quality  Program  is  to  establish  the  efficacy  of  existing 
professional  development  programs  for  teaching  mathematics  or  science  from 
pre-kindergarten  through  high  school  with  small  efficacy  or  replication  trials  and 
effectiveness  evaluations  of  interventions  implemented  at  scale.  Similarly,  an 
objective  of  the  National  Science  Foundation’s  Research,  Evaluation,  and 


2  As  part  of  its  mission  under  law  (the  Deficit  Reduction  Act  of  2006),  the  newly  formed  Academic  Competitiveness 
Council  will  evaluate  the  effectiveness  of  all  existing  federal  STEM  education  programs,  identifying  areas  of 
overlap  and  recommending  ways  to  efficiently  integrate  and  coordinate  them  in  the  future  (see 
http://www.ed.gov/news/pressreleases/2006/03/030620Q6.htmn. 
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Technical  Assistance  program  is  to  develop  new  models  and  tools  for 
documenting  Math  and  Science  Partnership  projects’  progress  toward  their  goals. 
And,  the  National  Science  Foundation’s  Evaluative  Research  and  Evaluation 
Capacity  Building  program  supports  projects  that  develop  unique  approaches  to 
evaluation  practice  in  the  generation  of  knowledge  for  the  STEM  education 
community. 

•  Rigorous  methods  and  designs 

During  the  past  several  years,  issues  concerning  the  appropriateness  of  research 
designs  and  methods  have  come  to  the  forefront  of  national  educational  policy 
and  research  discussions.  In  large  part,  increased  interest  in  this  topic  was 
generated  by  the  No  Child  Left  Behind  legislation’s  emphasis  on  “scientifically- 
based  research”  and  “evidence-based”  approaches  to  educational  practice.  While 
the  agencies  vary  to  some  extent  in  their  methodological  preferences,  there  is 
consensus  that  the  research  method(s)  employed  should  fit  the  question  at  hand 
(see  Section  II).  There  is  also  consensus  that  the  methodological  rigor  of  funded 
projects  is  increasing. 

•  Wide  array  of  both  quantitative  and  qualitative  methods 

As  discussed  in  Section  II,  the  agencies  support  a  wide  array  of  research 
methodologies  and  data- analytic  procedures. 

•  Enhanced  focus  and  selectivity  in  requests  for  applications 

Agencies  have  made  significant  strides  in  the  area  of  program  solicitations. 
Overall,  the  Task  Group  found  that  current  requests  for  applications  are  quite 
detailed  and  specific  with  regard  to  program  objectives,  preferred  methodologies, 
and  review  criteria.  However,  there  is  some  overlap  among  the  agencies  in 
funding  foci  and  some  confusion  in  the  field  about  where  to  go  for  funding  in 
various  areas.  Further  examination  of  this  situation  should  yield  specific 
recommendations  that  will  help  to  streamline  these  efforts  and  concomitantly 
produce  greater  clarity  for  the  research  community. 

•  Focus  on  role  of  technology 

A  significant  number  of  programs  focus  on  assessing  the  impact  of  using 
educational  technologies  designed  to  improve  students’  academic  achievement  in 
mathematics  and  science  (ED- 10,  NSF-2).  Several  of  these  programs  target 
students  with  disabilities  (ED-12,  ED-13,  ED-14). 

•  Development  of  national  databases 

Several  of  the  reports  analyzed  by  the  Task  Group  call  for  the  establishment  of 
national  databases  for  improving  the  teaching  of  science,  mathematics,  and 
technology.  A  number  of  databases  currently  exist  that  provide  a  range  of  useful 
information  in  this  area.  For  example,  the  ERIC  database  (www.eric.ed.gov) 
provides  a  centralized  bibliographic  database  of  journal  articles  and  other 
published  and  unpublished  education  materials.  The  new  version  of  ERIC  has 
added  numerous,  free  full-text  documents  for  immediate  download  at  no  charge 
and  also  provides  electronic  links  to  commercial  sources,  making  this  process 


42 


easy  to  use  and  up  to  date.  In  addition,  the  What  Works  Clearinghouse 
(http://www.whatworks.ed.gov/)  provides  educators,  policymakers,  and  the  public 
with  a  central  and  trusted  source  of  scientific  evidence  of  what  works  in 
education. 

Process  recommendations  requiring  further  attention 

•  Development  of  a  STEM  education  federal  investment  database 

This  proposed  database  would  provide  basic  information  on  all  federally  funded 
STEM  education  projects. 

•  Communication  of  research  results 

More  effective  ways  of  communicating  research  results  to  policy  makers  and 
educational  practitioners  are  needed.  While  project  websites  and  agency 
clearinghouses  are  a  significant  step  in  this  direction,  federal  educational  research 
programs  need  to  do  a  better  job  of  linking  research  with  practice.  Specific 
recommendations  are  provided  in  Section  III. 

•  Syntheses  of  available  knowledge 

Several  reports  noted  the  need  for  agencies  to  support  periodic  meta-analyses 
other  types  of  syntheses  of  important  topic  areas  in  order  to  make  the  knowledge 
base  more  accessible  to  practitioners  and  policy  makers. 

•  Links  among  research,  policy,  and  practice 

Much  needs  to  be  done  to  integrate  policy,  practice,  and  research  more  fully. 
Currently,  practitioners  rely  mainly  on  their  own  professional  judgment  or  that  of 
their  peers  when  deciding  whether  to  adopt  new  mathematics  or  science 
programs.  Until  research  findings  are  easily  accessible  and  considered  relevant, 
they  are  not  likely  to  draw  the  attention  of  school  principals  and  superintendents 
or  of  college  faculty.  However,  to  ensure  both  fidelity  of  implementation  and 
sustainability  of  effective  STEM  programs  in  entire  districts  and  states,  more 
research  is  needed  regarding  the  nature  of  large-scale  organizational  change  in 
educational  systems. 

•  Coordination  and  cooperation  among  federal  agencies 

Greater  coordination  and  cooperation  across  federal  agencies  is  needed.  For 
instance,  it  would  be  particularly  useful  to  create  a  federal  searchable  database  of 
all  STEM  education  and  evaluation  research  projects  being  supported  by  the 
various  agencies.  This  would  enable  practitioners  and  policy  makers  to  get  a 
better  sense  of  the  work  that  is  currently  being  funded  and  would  encourage 
greater  collaboration  across  the  various  programs.  Additional  recommendations 
for  enhancing  interagency  cooperation,  coordination,  and  collaboration  are 
provided  in  Section  VI. 


Section  VI:  Recommendations  for  Federal  Interagency  Cooperation, 
Coordination,  and  Collaboration 
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This  section  is  concerned  with  how  the  federal  agencies  can  work  together  to  increase 
effectiveness  and  cover  areas  with  significant  gaps  in  the  current  STEM  education  and 
evaluation  research  portfolio. 

On  June  5,  2003,  a  memo  was  sent  from  the  White  House  Office  of  Science  Technology  and 
Policy  and  the  Office  of  Management  and  the  Budget  to  Heads  of  Executive  Branch  Departments 
and  Agencies  focusing  on  FY  2005  Interagency  Research  and  Development  Priorities.  In 
discussing  education  and  workforce  development,  it  was  stated  that  “Effective  interagency 
collaboration  will  promote  the  implementation  of  research-based  programs  and  practices 
addressing  priorities  established  in  the  No  Child  Left  Behind  Act  of 2002 C  It  was  also  noted  that 
“  .  .  .  a  high  priority  will  be  given  to  research  addressing  math  and  science  learning  and 
achievement,  including  best  practices  for  improving  the  quality  of  math  and  science  education.” 

There  are  several  ways  in  which  federal  agencies  can  “work  together  to  ensure  that  gaps  in 
STEM  education  research  will  be  adequately  funded”: 

•  At  a  minimum,  agencies  should  keep  one  another  informed  of  the  solicitations  being 
planned  for  subsequent  fiscal  years. 

•  In  another  easily  accomplished  collaboration,  the  agencies  should  incorporate  into  their 
program  solicitations  references  to  funding  opportunities  in  the  other  agencies. 

•  The  agencies  could  maintain  a  common  database  of  all  of  their  funded  STEM  education 
research  projects,  perhaps  developing  and  using  a  common  structured  abstract  format. 

•  Going  one  step  further,  agencies  could  cooperate  by  discussing  the  domains  of  STEM 
education  research  where  each  agency  has  particular  strengths  and  then  agree  upon  how 
to  most  efficiently  collaborate  in  these  areas,  address  gaps,  and  avoid  duplication  of 
effort. 

•  The  agencies  could  collaborate  on  developing  and  publishing  solicitations  and  then  either 
independently  fund  the  grants  making  the  cut  for  the  final  funding  slate  or  else  co-fund 
these  grants. 

•  The  agencies  could  jointly  sponsor  evaluations  of  their  programs  in  specified  areas,  at 
least  where  the  program  goals  of  the  agencies  are  similar  enough  to  pennit  such  an 
approach  to  be  of  benefit. 

•  The  agencies  could  co-sponsor  workshops  and  symposia  on  key  issues  related  to  STEM 
education  research  and  evaluation. 

As  straightforward  as  these  suggestions  may  appear,  they  do  pose  some  challenges.  For  example, 
the  three  agencies  have  different  “cultures”  and  different  kinds  of  connections  to  and 
relationships  with  the  field.  To  some  extent,  the  agencies  have  different  stakeholders,  and  their 
programs  cover  different  bands  on  the  education  continuum  (e.g.,  K-12,  community  college, 
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undergraduate).  There  is  also  a  long  history  of  separate  institutional  goal  setting.  Despite  these 
differences,  cooperation  may  be  achieved  by  a  continuing  dialogue  among  these  agencies 
regarding  their:  a)  respective  strengths  and  priorities;  b)  strategic  long-term  plans;  and  c)  funding 
precedents  and  constraints  (legal,  institutional,  historical  or  otherwise). 

It  should  be  noted  at  this  point  that  the  three  agencies  have  already  begun  making  progress  with 
respect  to  some  of  the  recommendations  listed  above.  For  example,  in  conjunction  with  the 
Brookings  Institution,  the  NICHD,  IES,  and  NSF  co-sponsored  a  conference  in  September  of 
2005  on  algebraic  reasoning.  The  aims  of  this  conference  were  to:  a)  examine  what  is  known 
about  the  requisite  developmental  and  cognitive  competencies  for  proficient  pre-algebraic  and 
algebraic  reasoning,  and  how  best  to  address  the  gaps  in  our  knowledge  base;  and  b)  analyze 
what  kinds  of  math  problems  should  (or  should  not)  be  categorized  as  algebraic  in  content  from 
the  perspective  of  the  field  of  mathematics. 

Interagency  Education  Research  Initiative 

As  noted  above,  one  of  the  ways  that  the  agencies  could  work  together  to  address  gaps  in  STEM 
education  research  is  to  collaborate  on  developing  and  publishing  solicitations.  The  Interagency 
Education  Research  Initiative  (IERI)  has  undoubtedly  been  the  most  ambitious  attempt  to  date  to 
achieve  this  objective.  The  purpose  of  IERI  was  to  provide  opportunities  for  the  scaling  up  of 
“evidence-based”  interventions  in  mathematics,  science,  and  reading  as  they  are  implemented  in 
varied  school  contexts  with  diverse  student  populations  that  included  the  use  of  technology  as  a 
key  factor.  A  joint  program  solicitation  was  developed.  For  the  first  several  years,  NSF  managed 
the  competition  and  processed  the  awards  while  award  oversight  was  provided  collaboratively  by 
the  three  agencies,  and  agendas  for  the  annual  PI  meetings  were  jointly  developed  and  staffed. 
Although  this  effort  was,  by  and  large,  successful,  it  was  decided  that  2004  would  be  the  last 
year  for  this  collaborative  initiative,  and  that  each  agency  would  subsequently  attempt  to  provide 
opportunities  for  research  on  scaling  up  through  its  own  separate  solicitation  process.  There  were 
multiple  reasons  for  this: 

1 .  The  three  agencies  had  different  processes  for  reviewing  proposals  and  making 
awards. 

2.  NSF’s  mission  focused  exclusively  on  mathematics  and  science,  whereas  IES  and 
NICHD  made  the  funding  of  reading  studies  a  priority  as  well. 

3.  With  all  three  agencies  involved  in  making  key  decisions  regarding  the  IERI  program 
and  its  solicitations,  it  became  difficult  to  arrive  at  a  consensus  in  a  timely  manner. 

4.  Principal  Investigators  were  often  confused  by  the  agencies’  differing  processes  and 
priorities. 

Still,  much  was  learned  in  the  implementation  of  the  IERI  Program,  and  the  central  goal  of  that 
Program  remains  an  important  one  for  all  the  agencies  involved.  Furthermore,  the  obstacles  that 
prevented  the  continuation  of  IERI  are  not  insurmountable.  For  example,  advances  are  being 
made  with  respect  to  numbers  1  and  4,  above,  not  only  as  outlined  in  the  following  discussion  of 
recently  developed  guidelines  for  NIH-NSF  Extramural  Collaborations,  but  also  emanating  from 
the  NSTC’s  Research  Business  Models  Subcommittee  (see  http://rbm.nih.gov/).  The  purpose  of 
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this  Subcommittee  is  to  advise  and  assist  the  Committee  on  Science  and  the  NSTC  on  policies, 
procedures,  and  plans  relating  to  business  models  to  improve  the  efficiency,  effectiveness  and 
accountability  of  the  Federal  research  and  development  enterprise  in  a  manner  cognizant  of 
currently  available  resources.  Concomitantly,  eventual  adoption  by  all  federal  agencies  of  the 
SF-424  grant  application  form  (see  http://thefdp.org/SF424RR  Info.html)  should  greatly 
improve  interagency  collaborative  funding  efforts,  at  least  with  respect  to  addressing  problems 
currently  associated  with  the  lack  of  consistency  across  the  various  agency-specific  grant 
application  forms. 

With  respect  to  the  content  foci  of  IERI,  there  is  no  reason  why  a  future  interagency 
collaboration  could  not  limit  these  to  the  domains  of  mathematics  and  science.  As  to  the  focus  on 
“scaling  up”  of  promising  STEM  interventions,  more  research  is  certainly  needed  on  how  such 
efforts  would  play  out  at  all  educational  levels.  Finally,  although  scaling  up  might  certainly 
prove  beneficial  as  one  of  the  goals  of  such  an  initiative,  other  possible  avenues  of  collaboration 
may  also  be  worth  examining.  Attempting  to  explore  such  collaborative  and  cooperative 
directions  would  at  the  very  least  require  continuing,  interagency  assessment  of  STEM  education 
programs,  which  should  in  itself  enhance  the  efficiency  of  federal  investments  in  these  areas.  As 
noted  above,  however,  it  is  equally  evident  that  each  federal  agency  has  its  own  mission(s)  and 
culture.  Although  differences  in  this  regard  can  sometimes  hinder  collaborative  efforts  (e.g.,  due 
to  issues  of  territoriality),  they  can  also  yield  a  potentially  richer  and  more  efficient  redirection  of 
existing  resources  in  the  service  of  common  objectives. 

Toward  the  latter  end,  we  recommend  that  the  existing  NSTC  Education  and  Workforce 
Subcommittee  (or  one  of  its  working  groups)  serve  as  an  advisory  group  for  the  purpose  of 
revisiting  the  development  of  interagency  STEM  education  research  initiatives  in  specific  areas 
of  mutual  interest. 

Guidelines  for  NIH  -  NSF  Extramural  Collaborations 

Although  the  IERI  program  reflects  some  of  the  complexities  associated  with  interagency 
coordination,  it  should  be  noted  that  several  federal  agencies  have  been  working  hard  during  the 
past  few  years  to  overcome  some  of  these  difficulties.  For  example,  in  May  of  2005,  NIH  and 
NSF  arrived  at  an  agreement  on  a  set  of  guidelines  for  extramural  research  collaborations  (i.e., 
pertaining  to  scientific  research  in  general)  between  the  two  agencies.  To  this  end,  the  NIH-NSF 
Collaborations  Workgroup  prepared  a  “Toolkit  for  Interagency  Collaborations  to  Support 
Research  Grants.”  This  workgroup  points  out  that,  “Joint  efforts  among  various  federal  agencies 
may  be  developed  to  address  emerging  or  cross-cutting  scientific  areas,  or  to  support  large 
projects  that  benefit  researchers  supported  by  several  agencies.”  Furthermore,  they  note  that  the 
pooling  of  agency  resources  can  constitute  both  an  efficient  and  effective  means  of  dealing  with 
either  a  selected  or  multi-disciplinary  scientific  community.  Nevertheless,  they  point  out  that 
differences  in  the  regulations,  practices  and  procedures  of  the  participating  agencies  must  be 
resolved  ahead  of  time  if  smooth  coordination  of  solicitation,  review,  funding,  and  oversight 
activities  is  to  be  ensured.  Among  the  important  issues  to  be  worked  out  between  the  agencies 
are  the  format,  content,  and  timing  of  program  announcements,  the  required  format  and  content 
of  applications,  the  form  and  conduct  of  the  peer  review  process,  the  transfer  of  applications 
from  one  agency  to  another,  if  required,  and  the  oversight  of  awards.  The  specific  guidelines 
include  detailed  models  for  and  examples  of  the  following: 
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•  Memorandum  of  Understanding 

•  Special  Language 

•  Implementation  Plans 

•  Funding  Announcement 

•  Peer  Review  Process 

•  Scoring  Equivalence 

•  Transfer  of  Applications  Between  Agencies 

•  Grants  Management  Process 

These  guidelines  could  be  adapted  for  use  by  other  agencies  interested  in  collaborating  on 
educational  research  initiatives,  and  thus  are  likely  to  help  in  reducing  the  complexities 
associated  with  interagency  collaborations  that  often  inadvertently  prohibit  the  efforts  of  research 
administrative  staff  to  establish  the  cooperative  efforts  that  are  becoming  increasingly  crucial  for 
supporting  large-scale,  multidisciplinary  education  research  projects. 
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Section  VII:  Attracting  Qualified  Investigators  to  Undertake  Research  in 

STEM  Education 

Given  the  magnitude  of  education  problems  in  the  U.S.,  it  is  critical  that  education  research 
encompass  the  insights  from  a  multitude  of  fields  and  bring  the  analytic  approaches  from  these 
fields  to  bear  on  important  education  problems.  Some  disciplines  where  potential  expertise  in 
educational  research  resides  include  organizational  theory,  epidemiology,  economics, 
psychology,  the  physical  sciences,  engineering,  and  anthropology.  There  are  several  approaches 
that  can  be  taken  to  encourage  investigators  who  study  other  societal  issues  to  undertake  research 
on  education: 

a.  Require  the  use  of  interdisciplinary  teams  in  RFPs. 

The  IERI  program  solicitations  have  required  the  use  of  interdisciplinary  teams  in  examining 
issues  related  to  scaling  up  education  interventions  in  real  education  settings.  This  has 
resulted  in  interesting  cross-fertilization  of  research  questions  and  methods  and  has  brought 
together  experts  in  STEM  education,  education  psychology,  cognitive  science,  sociology, 
anthropology,  and  education  policy.  For  example,  one  IERI  project  that  is  studying  the 
scaling  up  of  highly  rated  science  programs  in  middle  schools  has  an  anthropologist  on  the 
research  team.  The  anthropologist  has  video-taped  one  table  of  four  students  in  a  science 
class  for  an  entire  year  and  is  analyzing  the  students’  interactions  with  each  other  and  the 
teacher,  the  kinds  of  questions  they  ask,  and  the  kinds  of  problem  solving  strategies  the 
science  tasks  elicit.  This  approach  greatly  enriches  the  other,  more  quantitative,  data  being 
collected  in  the  project. 

b.  Support  conferences  and  forums. 

Conferences  can  address  this  issue  head-on  through  commissioning  papers  and  discussions 
around  the  issue.  For  example,  the  NSF  funded  a  multi-methods  research  forum  that  the 
National  Academy  of  Sciences  convened  in  collaboration  with  the  American  Education 
Research  Association  and  the  American  Psychological  Association.  The  focus  of  the  forum 
was  to  explore  when  and  how  a  multi-methods  approach  in  researching  important  education 
issues  is  most  useful. 

c.  Conduct  active  outreach  to  communities  other  than  education. 

Program  staff  with  connections  to  various  disciplines  can  engage  in  outreach  through 
attending  annual  meetings,  describing  available  funding  opportunities,  and  providing 
technical  assistance  in  the  proposal  development  process.  In  addition,  members  of 
communities  outside  of  STEM  disciplines  and  education  can  be  invited  to  serve  on  review 
panels  or  as  rotating  program  officers. 
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Section  VIII:  Attracting  Qualified  Students  to  STEM  Education  and 

Evaluation  Research 

Attracting  and  training  a  new  generation  of  education  researchers  has  taken  on  new  urgency  as 
federal  legislation  (e.g.,  the  No  Child  Left  Behind  Act  of  2001  and  the  Education  Sciences 
Reform  Act  of  2002)  has  underscored  the  need  for  “scientifically  based  research”  in  education 
(Eisenhart  and  DeHaan,  2005).  Similarly,  calls  for  improving  graduate  training  in  education  are 
now  commonplace  (Institute  of  Education  Sciences,  2004;  National  Research  Council,  2002, 
2004;  Eisenhart  and  DeHaan,  2005). 

A  number  of  recent  reports  have  described  current  education  practice  as  not  resting  on  a  solid 
research  base  (Coalition  for  Evidence-Based  Policy,  2002;  National  Research  Council,  2002). 
Instead,  policy  decisions  are  oftentimes  based  on  personal  experience,  folk  wisdom,  and 
ideology.  Grounding  education  policy  and  practice  in  the  United  States  on  evidence  will  require 
a  transformation  of  the  field.  Practitioners  will  have  to  turn  routinely  to  education  research  when 
making  important  decisions,  and  education  researchers  will  have  to  produce  research  that  is 
relevant  to  those  decisions.  To  achieve  this  goal,  there  is  a  need  for  a  cadre  of  well-trained 
researchers  capable  of  conducting  high  quality  research  that  is  relevant  to  practitioners  and 
policymakers  (Institute  of  Education  Sciences,  2004). 

There  are  significant  capacity  issues  within  the  education  research  and  evaluation  fields. 
According  to  a  survey  conducted  by  the  National  Opinion  Research  Center,  only  7%  of  doctorate 
recipients  in  the  field  of  education  cite  research  and  development  as  their  primary  postdoctoral 
activity  (Hoffer  et  al.  2003).  Similarly,  a  membership  survey  conducted  by  the  American 
Education  Research  Association  (AERA)  revealed  that  less  than  a  quarter  of  its  membership 
cites  research  as  being  their  major  responsibility  (AERA,  2002). 

There  are  also  significant  issues  pertaining  to  the  nature  of  the  training  that  is  currently  being 
provided  by  graduate  programs.  New  researchers  are  not  being  exposed  to  all  the  skills  and 
training  they  need  at  the  graduate  level  to  do  the  type  of  rigorous  research  that  is  needed 
(Eisenhart  &  DeHaan,  2005). 

To  initiate  the  changes  necessary  for  the  development  of  a  cadre  of  well-trained  STEM  education 
research  and  evaluation  experts,  we  make  the  following  recommendations: 

1.  Establish  Innovative  Pre-Doctoral  and  Post-Doctoral  Training  Programs  in 
STEM  Education  and  Evaluation  Research.  Training  programs  such  as  the 
Institute  of  Education  Sciences  Pre-Doctoral  Interdisciplinary  Research  Training 
Program  in  the  Education  Sciences  (IES,  2004)  provide  a  model  in  this  area.  The  goal 
of  this  program  is  to  increase  the  supply  of  scientists  and  researchers  in  education 
who  are  well-trained  to  conduct  a  new  generation  of  rigorous  evaluation  studies, 
develop  new  products  and  approaches  to  education  which  are  grounded  in  the  science 
of  learning,  design  valid  assessments  and  measures,  and  explore  data  with 
sophisticated  statistical  techniques.  During  fiscal  years  2004  and  2005,  grants  were 
awarded  to  institutions  that  proposed  a  rigorous  and  coherent  program  of  study  across 
disciplines  such  as  education,  psychology,  economics,  statistics,  political  science, 
human  development,  and  epidemiology.  The  overarching  goal  of  this  program  is  to 
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produce  a  cadre  of  well-trained  education  researchers  that  are  capable  of  and  willing 
to  conduct  high  quality  research  that  addresses  pressing  problems  and  challenges  in 
American  education. 

2.  Establish  an  Early  Career  STEM  Education  Research  Grant  Program.  The  goal 
of  this  program  would  be  to  fund  research  carried  out  by  Principal  Investigators  who 
are  in  the  early  stages  of  their  careers  (e.g.,  within  5  years  of  having  been  awarded 
their  Ph.D.).  While  a  number  of  federal  agencies  currently  provide  some  form  of  early 
career  or  career  development  research  award,  for  the  most  part,  these  are  not  targeted 
specifically  for  STEM  education  research. 

3.  Include  projects  in  the  portfolio  such  as  centers  that  combine  support  for 
graduate  students,  faculty  with  different  kinds  of  expertise,  and  opportunities 
for  collaboration  across  fields  and  institutions.  Centers,  such  as  NSF’s  Centers  for 
Learning  and  Teaching  and  the  Science  of  Learning  Centers  provide  opportunities  for 
students  from  diverse  fields  such  as  the  STEM  disciplines,  education  research,  and 
cognitive  science  to  focus  on  a  common  research  agenda.  The  National  Institute  of 
Child  Health  and  Human  Development’s  (NICHD)  Multidisciplinary  Research 
Centers  in  Learning  Disabilities  offer  similar  opportunities.  As  an  example  of  the 
value  added  of  such  interdisciplinary,  collaborative  efforts,  both  the  NSF  and  NICHD 
Center  programs  are  supporting  research  on  the  cutting  edge  of  cognitive 
neuroscience,  where  advances  in  neuroimaging  and  related  methodologies  are 
yielding  important  insights  into  the  brain-based  correlates  of  instructional  practices. 
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Concluding  Observations  and  Future  Directions 

A  number  of  noteworthy,  STEM  education-related  reports  have  been  published  and  federally- 
organized  conferences  held  during  the  past  year  since  completion  of  the  Task  Group’s  analyses. 
As  several  of  these  have  direct  relevance  for  the  issues  treated  in  the  present  report,  their  main 
findings  and  implications  will  be  discussed  briefly  in  this  section.  First,  the  results  of  a  survey 
conducted  by  Education  Insights  at  Public  Agenda,  an  opinion  research  organization,  was 
released  in  a  report  entitled  “ Reality  Check  2006:  Are  American  Parents  and  Students  Ready  for 
More  Math  and  Science ?"  (http://www.publicagenda.org/research/pdfs/rc060 1  .pdf).  Among  the 
most  disturbing  findings  is  that  while  parents  tend  to  support  proposals  for  making  high  schools 
more  competitive  on  a  global  scale,  57%  also  believe  that  the  amount  of  math  and  science  their 
children  are  currently  being  taught  is  satisfactory  -  and  70%  of  parents  of  high  school  students 
hold  this  view.  Furthermore,  this  survey  (based  on  responses  from  telephone  interviews  with  a 
national  random  sample,  along  with  two  focus  groups)  revealed  that  the  concerns  which  parents 
expressed  about  math  and  science  achievement  in  1994  have  declined  over  the  past  decade.  That 
is,  whereas  48%  of  parents  in  1994  believed  their  children  were  not  getting  enough  math  and 
science  in  school,  only  32%  of  parents  in  2005  expressed  the  same  concern.  Even  more 
disconcerting  is  the  finding  that  despite  the  generally  acknowledged  importance  of  the  role  that 
science  and  technology  will  play  in  the  economy  of  the  future,  45%  of  American  students 
indicated  they  would  be  quite  unhappy  if  they  ended  up  in  a  job  or  career  which  required  them  to 
do  a  lot  of  math  or  science. 

To  many  STEM  educators,  such  results  may  not  seem  particularly  surprising.  However,  these 
findings  make  it  painfully  clear  that  even  if  federal  efforts  to  strengthen  the  STEM  education 
research  portfolio  and  improve  dissemination  of  findings  to  practitioners  and  policymakers  were 
to  make  considerable  progress,  the  ultimate  success  at  impacting  student  achievement  and  career 
patterns  would  probably  not  be  realized  unless  or  until  additional  efforts  are  directed  toward 
convincing  both  parents  and  students  of  the  importance  of  STEM  education.  Coming  up  with 
viable  approaches  to  achieve  these  changes  will  no  doubt  require  a  much  more  coordinated  effort 
by  the  scientific  community  itself,  not  just  the  STEM  education  community.  Although  the  Task 
Group  did  not  tackle  this  issue  in  the  present  report,  the  results  described  here  make  it  rather 
obvious  that  we  are  in  urgent  need  of  developing  evidence-based  methods  for  how  best  to  help 
the  American  public  become  more  cognizant  of  the  growing  importance  of  STEM  education  for 
the  future  economic  prosperity  of  this  country. 

In  many  of  the  programs  reviewed  in  the  present  document,  the  vast  majority  of  research  efforts 
are  aimed  at  improving  STEM  education  for  school-age  (K-12)  students  “in  general,”  while 
others  are  directed  toward  equity  issues,  including  gender,  race/ethnicity,  SES,  and  children  with 
learning  difficulties.  However,  rarely  do  research-based  programs  focus  on  issues  pertaining  to 
the  identification  of  children  who  may  be  exhibit  the  potential  for  developing  unique  talents  in 
one  or  more  of  the  STEM  areas  and  could  possibly  go  on  to  become  scientific  leaders  and 
innovators.  Consequently,  a  planning  meeting  on  Identifying  and  Developing  Talent  in  STEM 
was  recently  (September  2006)  held  at  The  National  Academies  Center  for  Education,  co¬ 
sponsored  by  the  U.S.  Department  of  Education,  the  National  Institutes  of  Health,  the  National 
Science  Foundation,  and  the  American  Psychological  Association.  Several  questions  were 
designed  to  guide  the  presentations  and  discussions  at  this  meeting  including:  How  is  talent  in 
STEM  currently  identified?  How  might  STEM  talent  be  identified  in  ways  that  would  expand 
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and  diversity  the  talent  pool?  Is  talent  domain  specific?  Is  it  based  on  core  abilities,  and  if  so, 
what  are  these?  What  evidence  is  available  to  address  these  questions?  What  are  the  implications 
for  research,  policy,  and  practice?  In  an  effort  to  bridge  some  of  the  communication  gaps 
between  research,  practice  and  policy  alluded  to  earlier  in  this  report,  this  conference  brought 
together  researchers,  policymakers  and  practitioners  in  an  effort  to  begin  a  dialogue  that  will  set 
the  stage  for  future  meetings  and  perhaps  the  development  of  more  coordinated  research 
solicitations  downstream  that  will  include  input  from  various  types  of  stakeholders  early  on  in 
the  process.  In  any  event,  a  more  focused  effort  by  funding  agencies  on  developing  an 
evidenced-based  approach  to  designing  STEM  talent  initiatives  in  the  middle  and  secondary 
school  years  would  no  doubt  yield  some  very  important  benefits  for  increasing  the  STEM  talent 
pool  in  this  country. 

In  May  of  2005,  the  National  Academies  received  a  congressional  request  to:  a)  come  up  with 
the  top  10  actions  that  federal  policymakers  could  take  to  improve  the  science  and  technology 
enterprise  so  that  the  U.S.  could  successfully  compete  and  prosper  in  the  global  community  of 
the  21st  century;  and  b)  provide  a  strategy,  with  several  concrete  steps,  that  could  be  used  to 
implement  each  of  the  recommended  actions.  In  October  of  2005,  in  response  to  this  request,  a 
report  entitled  “Rising  Above  the  Gathering  Storm:  Energizing  and  Employing  America  for  a 
Brighter  Economic  Future”  was  released  (http://www.nap.edu/books/0309100399/html). 

Of  the  report’s  four  major  recommendations,  the  one  most  relevant  to  the  topics  treated  in  the 
present  document  is  that  America’s  talent  pool  should  be  increased  by  vastly  improving  K-12 
mathematics  and  science  education.  Although  the  specific  “implementation  actions”  provided  in 
the  National  Academies  report  are  based  on  the  extant  U.  S.  educational  model,  the  study 
committee  acknowledged  that  some  commentators  have:  a)  questioned  whether  educational 
reform  based  on  this  model  is  capable  of  producing  effective  and  long-lasting  improvements,  and 
b)  suggested  that  it  is  challenging  to  recommend  “tried  and  true”  programs,  given  the  lack  of  a 
well-developed  literature  on  the  effectiveness  of  K-12  teaching  and  learning  interventions. 
Although  the  report  goes  on  to  recommend  the  adoption  of  several  existing  K-12  programs,  it 
acknowledges  “.  .  .we  must  emphasize  the  need  for  research  and  evaluation  to  serve  as  a 
foundation  for  change  in  K-12  mathematics  and  science  education”  (p.  94). 

We  wholeheartedly  endorse  the  assertion  that  research  and  evaluation  can  provide  a  strong 
foundation  for  education  reform.  Furthermore,  we  hope  that  the  analyses  provided  in  the  present 
report  will  assist  federal  agencies  in  their  continuing  efforts  to  strengthen  the  federal  STEM 
education  research  portfolio,  so  that  future  recommendations  for  improving  instructional 
practices,  student  achievement,  and  professional  development  in  STEM  education  will  indeed  be 
founded  on  a  rigorous  evidence  base. 
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*The  Task  Group  members  are  grateful  to  Ann  Carlson,  Senior  Staff  Associate  for  Policy  and  Planning  in  NSF’s  Office  of  the 
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Jones,  Deputy  Associate  Director  of  the  Science  Division  at  OSTP  provided  extremely  valuable  advice,  both  substantive  and 
editorial,  during  the  final  preparation  of  the  report.  Thanks  are  also  extended  to  Judith  A.  Ramaley,  fonner  Assistant  Director  of 
EHR  at  NSF,  for  her  noteworthy  guidance  of  this  effort  from  its  inception,  as  well  as  to  both  Lynn  Okagaki,  Commissioner  for 
Education  Research  at  IES,  and  Mona  Rowe,  Associate  Director  for  Science  Policy,  Analysis  and  Communication  at 
NICHD,  for  their  wise  counsel  and  editorial  assistance  throughout  the  development  and  writing  of  this  report. 
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U.S.  Department  of  Education 

The  programs  and  portfolios  listed  in  Table  1  include  virtually  all  of  the  projects  pertaining  to 
STEM  education  and  evaluation  research  funded  by  the  Department  of  Education  in  FY  2003. 
However,  several  exceptions  should  be  noted.  First,  projects  funded  through  the  Small  Business 
Innovation  Research  (SBIR)  program  were  not  included  in  the  Department  of  Education  portion 
of  Table  1,  nor  are  the  objectives  of  the  SBIR  program  listed  in  the  Department  of  Education 
portion  of  Table  3.  The  vast  majority  of  these  projects  are  designed  to  yield  a  commercial 
product  and  until  recently,  many  did  not  include  a  rigorous  evaluation  or  research  component. 

Second,  it  is  possible  that  some  additional  evaluation  projects  were  funded  through  other,  mostly 
non-research,  programs.  An  example  of  a  program  falling  in  this  category  is  the  Department  of 
Education’s  Mathematics  and  Science  Partnership  (MSP)  program 

(http://www.ed.gov/programs/mathsci/index.html).  The  main  objective  of  this  program  is  to 
increase  the  academic  achievement  of  students  in  mathematics  and  science  by  enhancing  the 
content  knowledge  and  teaching  skills  of  classroom  teachers.  Partnerships  between  high-need 
school  districts  and  the  science,  technology,  engineering,  and  mathematics  (STEM)  faculty  in 
institutions  of  higher  education  are  at  the  core  of  these  improvement  efforts.  The  Mathematics 
and  Science  Partnerships  program  is  a  formula  grant  program  to  the  states,  with  the  size  of 
individual  state  awards  based  on  student  population  and  poverty  rates.  Each  State  is  responsible 
for  administering  a  competitive  grant  competition,  in  which  grants  are  made  to  partnerships  to 
improve  teacher  knowledge  in  mathematics  and  science.  Although  these  grants  now  include  a 
small  evaluation  component,  these  projects  were  not  considered  to  fall  under  the  purview  of 
“research”  as  defined  for  this  report.  As  such,  the  dollars  spent  on  this  program  in  FY  2003  are 
not  included  in  the  Department  of  Education  portion  of  Table  2,  nor  are  the  program  objectives 
listed  in  the  Department  of  Education  portion  of  Table  3. 

In  an  effort  to  provide  the  most  comprehensive  list  possible,  Program  Officers  at  the  Institute  of 
Education  Sciences  and  at  the  Office  of  Special  Education  and  Rehabilitative  Services  (OSERS) 
were  asked  to  examine  their  respective  portfolios  and  identify  relevant  projects  in  the  area  of 
STEM  education  and  evaluation  research.  Program  Officers  provided  the  members  of  the  Task 
Group  a  list  of  these  projects  or  programs  with  the  corresponding  funding  amounts  for  FY  2003. 
Although  a  number  of  projects  were  forward-funded,  we  only  report  in  Table  2  the  amounts  that 
were  budgeted  for  FY  2003.  In  addition,  requests  for  applications  for  the  identified  programs 
were  collected  and  their  objectives  analyzed. 

It  is  important  to  note  that  the  Individuals  with  Disabilities  Education  Improvement  Act  of  2004 
transferred  the  responsibilities  for  research  in  special  education  within  the  U.S.  Department  of 
Education  from  the  Office  of  Special  Education  and  Rehabilitative  Services  to  the  Institute  of 
Education  Sciences.  As  noted  earlier  in  this  report,  the  new  National  Center  for  Special 
Education  Research  is  now  a  functioning  organization  that  has  taken  on  the  responsibilities  for  a 
new  generation  of  special  education  research.  As  this  report  was  being  edited,  several  new 
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research  grant  competitions  focusing  on  Mathematics  and  Science  Special  Education  Research 
and  Special  Education  Teacher  Quality  Research  in  Mathematics  and  Science  were  launched 
(http://www.ed.gov/about/offices/list/ies/programs.html).  These  new  entries  are  not  included  in 
the  U.S.  Department  of  Education  portion  of  Table  3. 

National  Science  Foundation 


While  all  six  of  NSF's  Directorates  fund  education  outreach,  the  particular  mission  of  the 
Education  and  Human  Resources  (EHR)  Directorate  is  to  provide  leadership  in  the  effort  to 
improve  STEM  education.  Thus  EHR  has  by  far  has  the  most  extensive  array  of  programs  that 
fund  education  research  in  STEM.  The  programs  listed  in  Table  1  include  all  of  the  programs 
with  identifiable  education  research  components  in  NSF's  Directorate  of  Education  and  Human 
Resources  (EHR)  and  in  the  Directorate  of  Social,  Behavioral  and  Economic  Sciences  (SBE)  for 
FY  2003.  These  two  Directorates  have  responsibility  for  both  education  and  education  research 
activities. 

In  an  effort  to  locate  as  many  projects  as  possible  within  each  of  the  divisions  in  EHR,  program 
officers  were  asked  to  identify  all  the  program  solicitations  that  support  research  as  a  component. 
Although  individual  projects  supported  by  some  programs  in  both  EHR  (for  example,  the 
Mathematics  and  Science  Partnerships  or  Instructional  Materials  Development  projects)  and  in 
other  Directorates  can  include  an  education  research  component,  these  projects  were  not 
considered  for  Table  1.  Since  NSF  concerns  itself  completely  with  STEM  issues,  identifying 
each  one  of  these  individual  projects  would  have  been  prohibitive.  In  that  sense,  the  list  in  Table 
1  should  not  be  viewed  as  all-inclusive.  Nevertheless,  it  certainly  constitutes  a  reasonable 
estimate  of  NSF's  STEM  education  research  expenditures  for  FY  2003.  NSF's  Research  and 
Related  Activities  (R  and  RA)  Directorates  fund  education  activities  and  conduct  education 
outreach,  but  they  do  not  fund  education  research. 

In  order  to  generate  the  list  included  in  Table  1,  the  NSF  Task  Force  members: 

1 .  reviewed  all  the  program  solicitations  in  EHR  and  SBE  to  identify  programs  with 
education  research  as  a  component. 

2.  verified  with  Program  Officers  in  EHR  and  SBE  that  the  education  research  components 
were  operative  in  FY  2003. 

3.  asked  Program  Officers  associated  with  each  program  incorporating  an  education 
research  component  to  provide  funding  amounts  spent  on  education  research  in  FY  2003. 

4.  had  the  Program  Officers  review  Table  2  results  for  accuracy. 

5.  took  into  account  forward  funded  efforts. 


National  Institutes  of  Health 


The  funding  levels  and  program  objectives  for  the  NIH  provided  in  Tables  2  and  3,  respectively, 
represent  solely  the  portfolios  of  the  Mathematics  and  Science  Cognition  and  Learning  Program 
at  the  National  Institute  of  Child  Health  and  Human  Development  (NICHD).  Although  some  of 
the  other  Institutes  and  Centers  comprising  the  NIH  fund  research  projects  related  to  various 
facets  of  mathematical  and  scientific  cognition  and  learning,  these  are  primarily  investigator 
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initiated  grants  that  fall  under  the  rubric  of  more  generic  portfolios  in  domains  such  as  cognitive 
psychology,  cognitive  neuroscience,  sociology  and  demography,  population  studies,  etc. 
However,  the  National  Institute  of  Child  Health  and  Human  Development  houses  the  only 
program  at  the  NIH  devoted  specifically  to  the  funding  of  cognitive  and  educational  research  in 
mathematics  and  science  learning. 

Although  the  NICHD  program  described  above  is  the  sole  NIH  program  depicted  in  this  report,  it 
is  important  to  point  out  here  that  a  myriad  of  other  education  projects  are  funded  each  year  by 
the  NIH  that  are  primarily  concerned  with  biomedical  and  behavioral  science  education.  Indeed, 
the  NIH  has  an  office  expressly  devoted  to  education:  the  Office  of  Science  Education  (OSE) 
which  is  housed  in  the  Office  of  the  Director  of  the  NIH.  Although  the  OSE  does  not  fund 
research  in  science  education,  its  stated  mission  is  to  coordinate  a  comprehensive,  trans-NIH 
science  education  program  for  attracting  young  people  to  biomedical  and  behavioral  science 
careers  and  for  improving  science  literacy  in  both  children  and  adults.  Among  other  activities, 
the  OSE:  a)  develops,  supports,  and  directs  new  program  initiatives  at  all  levels,  with  special 
emphasis  on  K-16  students,  their  teachers  and  parents,  and  the  general  public;  b)  advises  the  NIH 
leadership  on  science  education  issues;  c)  examines  and  evaluates  research  and  emerging  trends 
in  science  education  and  science  literacy  for  policy  making;  d)  coordinates  science  education 
efforts  across  the  NIH  by  working  the  intramural,  extramural,  women's  health,  laboratory  animal 
research,  and  minority  program  offices  on  relevant  issues  and  programs;  e)  works  with  NIH 
institutes,  centers,  and  divisions  to  enhance  communication  of  science  education  activities;  and  f) 
works  cooperatively  with  other  public-  and  private-sector  organizations  to  develop  and 
coordinate  science  education  activities.  For  information  about  the  NIH  Office  of  Science 
Education,  see  http://science.education.nih.gov. 

As  part  of  its  science  literacy  and  education  efforts,  the  NIH  has  also  been  supporting  the 
Science  Education  Partnership  Awards  (SEP A)  for  close  to  15  years.  These  grants  bring  together 
biomedical  and  behavioral  researchers,  educators,  community  groups,  and  other  interested 
organizations  in  partnerships  to  create  and  disseminate  programs  that  give  K-12  students  and 
teachers  and  the  general  public  a  better  understanding  of  life  sciences.  The  Division  for  Clinical 
Research  Resources  (DCRR)  of  the  National  Center  for  Research  Resources  (NCRR)  oversees 
the  SEPA  Program,  which  funds  science  centers  and  museums  across  the  country.  Several 
academic  institutions  have  formed  partnerships  with  these  science  museums  and  centers,  using 
their  information  networks  to  develop  both  stationary  and  traveling  exhibits  on  fundamental 
biology  and  related  topics.  SEPA  funding  provides  biomedical  and  clinical  researchers  with  a 
vehicle  for  conveying  their  knowledge  as  well  as  their  appreciation  of  scientific 
accomplishments.  Moreover,  a  website  is  being  established  specifically  to  provide  access  to  the 
educational  expertise  and  materials  produced  via  these  efforts.  For  more  general  information 
about  the  SEPA  Program,  see  http://www.ncrr.nih.gov/clinical/cr  sepa.asp. 

Although  many  of  the  SEPA  grants  have  evaluation  components,  these  projects  were  not 
considered  to  fall  under  the  purview  of  “research”  as  defined  for  this  report.  As  such,  the  dollars 
spent  on  this  program  in  FY  2003  are  not  included  in  the  NIH  portion  of  Table  2,  nor  are  the 
program  objectives  listed  in  the  NIH  portion  of  Table  3. 

It  is  also  important  to  note  that  the  Minority  Access  to  Research  Careers  (MARC)  Program 
Branch  of  the  Division  of  Minority  Opportunities  in  Research  (MORE)  of  the  National  Institute 
of  General  Medical  Sciences  (NIGMS)  at  the  NIH  funds  research  training  opportunities  for 
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students  from  minority  groups  that  are  underrepresented  in  the  biomedical  and  related  sciences, 
including  mathematics.  The  MARC  Program  provides  support  for  stimulating  the  interest  of 
underrepresented  minority  students  to  consider  a  career  in  biomedical  research.  Furthermore,  the 
MARC  Branch  supports  the  Post-Baccalaureate  Research  Education  Program  (PREP)  initiative, 
designed  to  encourage  underrepresented  minorities  who  hold  a  recent  baccalaureate  degree  in 
biomedically  relevant  sciences  to  pursue  a  doctoral  degree.  For  more  information  about  the 
MARC  Program  Branch,  see  http://www.nigms.nih.gov/minority/marc.html. 

Finally,  it  should  be  pointed  out  that  the  NIH  provides  research  supplements  to  Principal 
Investigators  holding  specific  types  of  NIH  research  grants  in  order  to  improve  the  diversity  of 
the  research  workforce  by  recruiting  and  supporting  students,  postdocs,  and  eligible  investigators 
from  underrepresented  groups. 

Federal  STEM  Education  Programs  Not  Included  in  the  Present  Analysis 

In  this  section  we  provide  brief  descriptions  of  several  comparatively  large,  federal  STEM 
education  programs  that  were  not  included  in  the  present  analysis.  The  reader  should  be  apprised 
that  this  list  is  not  meant  to  be  comprehensive.  Rather,  it  comprises  a  sample  of  federal 
department  and  agency  STEM  education  programs  whose  projects  were  not  considered 
education  “research”  activities  as  stipulated  earlier  (see  Section  I),  while  nonetheless  constituting 
very  worthwhile  endeavors. 

National  Aeronautics  and  Space  Administration 

The  National  Aeronautics  and  Space  Administration  (NASA)  is  well  known  for  having  a  long 
and  distinguished  history  of  supporting  relevant  STEM  education  programs,  and  is  widely 
respected  as  a  leader  among  federal  agencies  in  such  efforts.  The  two  main  goals  of  NASA's 
education  program  are  to  “inspire  and  motivate  students  to  pursue  careers  in  science,  technology, 
engineering,  and  mathematics"  by  supporting  education  in  the  Nation's  schools  and  to  "engage 
the  public  in  shaping  and  sharing  the  experience  of  exploration  and  discovery”  by  supporting 
informal  education  and  public  outreach  efforts.  NASA's  commitment  to  education  places  special 
emphasis  on  these  goals  by:  a)  increasing  elementary  and  secondary  education  participation  in 
NASA  programs;  b)  enhancing  higher  education  capability  in  science,  technology,  engineering, 
and  mathematics  (STEM)  disciplines;  c)  increasing  participation  by  underrepresented  and 
underserved  communities;  and  d)  expanding  e-Education;  and  expanding  NASA's  participation 
with  the  informal  education  community. 

NASA  funds  a  vast  array  of  faculty  education  programs  (see 

http://education.nasa.gov/edprograms/frprograms/index.html).  However,  none  of  the  projects 
supported  by  these  programs  were  included  in  the  present  analysis,  as  they  do  not  focus  on 
STEM  education  research  as  defined  for  the  purposes  of  this  report.  This  is  not  to  say  that  the 
activities  supported  by  many  NASA  grants  lack  an  evaluation  component.  Indeed,  many  of  their 
programs  specify  the  need  for  an  evaluative  component,  in  which  the  success  of  a  given  project 
must  be  measured  in  terms  of  outcomes  such  as  the  audiences  served,  the  number  of  people 
reached,  and  the  impact  on  the  STEM  competencies  acquired  by  students.  However,  as  these 
approaches  do  not  constitute  “evaluation  research”  as  defined  in  this  report,  they  were  not 
included  in  the  present  analysis. 
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U.S.  Department  of  Energy 

The  Department  of  Energy  offers  a  variety  of  science-related  programs  for  teachers  and  students, 
including  educational  resources,  scholarships  and  internships,  contests  and  competitions,  and 
funding  for  schools  and  universities  (http://www.energv.gov/foreducators.htm).  Among 
numerous  other  activities  and  resources,  The  Department  oversees  a  variety  of  contests  and 
competitions  (http ://www. energy. gov/contests&competitions .htm),  and  also  runs  the  Oak  Ridge 
Institute  for  Science  and  Education  (http://www.orau.gov/orise.htm).  While  these  efforts  are 
indeed  laudable,  none  of  the  projects  supported  by  these  programs  were  included  in  the  present 
analysis,  because  they  do  not  focus  on  STEM  education  research  as  defined  in  this  report. 

U.S.  Department  of  Agriculture 

The  U.S.  Department  of  Agriculture  likewise  supports  many  commendable  science  education 
and  outreach  efforts 

(http://www.usda.gOv/wps/portal/lut/p/  s.7  0  A/7  0  10B?navtype=SU&navid=EDUCATI0N 

OUTREACH).  For  example,  they  provide  a  National  Agricultural  Library  with  educational 
resources  for  teachers  and  parents  (http://www.nal.usda.gov/outreach/resources.htm).  Again, 
however,  as  most  of  these  programs  are  not  of  research-based,  their  information  was  not  included 
in  the  present  analyses. 

Other  Non-Research  Based  Federal  Efforts  in  STEM  Capacity  Building 

As  depicted  in  Table  3,  a  variety  of  programs  dealing  with  aspects  of  “capacity  building”  have 
been  examined  for  the  present  report.  The  four  components  that  were  studied  include:  capacity 
building  for  STEM  education  research,  faculty  development,  teacher  development,  and  STEM 
workforce  improvement.  However,  it  should  be  noted  that  all  such  efforts  illustrated  in  this  table 
reflect  “research-based”  approaches  to  these  domains.  As  described  earlier,  numerous  programs 
at  the  federal  agencies  treated  in  this  report  as  well  as  at  other  federal  agencies  are  putting  a  great 
deal  of  effort  into  capacity  building.  These  non-research-based,  STEM  capacity  building 
activities  are  currently  being  examined  by  the  Human  Capacity  in  STEM  Working  Group  under 
the  auspices  of  the  NSTC  Education  and  Workforce  Development  Subcommittee,  and  will  be 
reported  on  at  a  later  date. 
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Specific  Objectives  of  Federal  STEM  Education  Research  Programs  —  Sorted  By  Category 


Category  Program  Code 


Program  Objectives 

(derived  from  published  program  authority3) 


Capacity  Building 

ED- 11 

To  develop,  implement,  and  evaluate  models  (including  models  for  professional  development)  for  infants, 
toddlers,  children,  or  youth  with  disabilities. 

Capacity  Building 

ED- 15 

To  establish  a  network  of  training  programs  that  collectively  produce  a  cadre  of  education  researchers  willing 
and  able  to  conduct  a  new  generation  of  methodologically  rigorous  and  educationally  relevant  scientific  research 
that  will  provide  solutions  to  pressing  problems  and  challenges  facing  American  education. 

Capacity  Building 

ED-15 

To  support  the  development  of  innovative  interdisciplinary  training  programs  for  doctoral  students  interested  in 
conducting  applied  education  research. 

Capacity  Building 

ED- 16 

To  produce  a  cadre  of  education  researchers  willing  and  able  to  conduct  a  new  generation  of  methodologically 
rigorous  and  educationally  relevant  scientific  research  that  will  provide  solutions  to  pressing  problems  and 
challenges  facing  American  education 

Capacity  Building 

ED-16 

To  increase  the  supply  of  scientists  and  researchers  in  education  who  are  prepared  to  conduct  rigorous  evaluation 
studies,  develop  new  products  and  approaches  that  are  grounded  in  a  science  of  learning,  design  valid  tests  and 
measures,  and  explore  data  with  sophisticated  statistical  methods. 

Capacity  Building 

ED- 16 

To  support  the  training  of  postdoctoral  fellows  interested  in  conducting  applied  education  research. 

Capacity  Building 

NSF-01 

To  build  capacity  (people  and  tools)  in  evaluation. 

Capacity  Building 

NSF-11 

To  increase  and  diversify  the  cadre  of  national  leaders  of  K-12  STEM  education  through  innovative  programs  for 
doctoral  and  postdoctoral  students. 

Capacity  Building 

NSF-11 

To  have  each  Center  organize  around  a  significant  national  question  and  provide  doctoral  and  post-doctoral 
programs  around  the  priority  area. 

Capacity  Building 

NSF-14 

To  enable  research  communities  that  can  capitalize  on  new  opportunities  and  discoveries  and  respond  to  new 
challenges. 

Capacity  Building 

NSF-14 

To  provide  innovative  educational,  research,  and  career  development  opportunities  for  all  center  participants. 

Curriculum  or  Instructional 
Practice 

ED-01 

To  conduct  complementary  research  studies  and  a  cross-site  program  evaluation  including  studies  that  address 
how  individual  or  background  differences  in  children  interact  with  the  curriculum  to  influence  developmental 
outcomes 

Curriculum  or  Instructional 
Practice 

ED-01 

To  conduct  complementary  research  studies  and  a  cross-site  program  evaluation  including  studies  that  compare 
different  versions  of  the  curriculum  or  different  approaches  to  implementation  in  order  to  identify  key  features  of 
the  curriculum  and  approaches  that  might  improve  effectiveness  and  ease  of  implementation. 

Curriculum  or  Instructional 
Practice 

ED-01 

To  determine  whether  one  or  more  curricula  produce  educationally  meaningful  effects  on  children. 

3  Program  authority  includes  Requests  for  Proposals/ Applications,  Program  Announcements,  other  types  of  grant-related  solicitations,  legislative  language,  and  web-based  program 
descriptions. 


Curriculum  or  Instructional 
Practice 

ED-03 

To  construct  learning  environments  that  exemplify  current  research  and  theory  about  effective  learning  of  math 
and  science. 

Curriculum  or  Instructional 
Practice 

ED-05 

To  develop  instruction  practice  or  materials  based  on  general  principles  of  learning  and  information  processing 
gained  from  cognitive  science  and  test  the  effects  of  these  new  approaches  within  education  delivery  settings. 

Curriculum  or  Instructional 
Practice 

ED-06 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will  result  in 
improving  mathematics  achievement  for  all  students. 

Curriculum  or  Instructional 
Practice 

ED-06 

To  support  the  development  of  new  interventions  and  approaches  to  mathematics  and  science  education  that  will 
eventually  result  in  improving  mathematics  and  science  achievement. 

Curriculum  or  Instructional 
Practice 

ED-07 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching  and  student 
learning  in  language  arts  and  mathematics. 

Curriculum  or  Instructional 
Practice 

ED-07 

To  study  how  the  links  between  knowledge,  reforms  and  practice  can  be  strengthened. 

Curriculum  or  Instructional 
Practice 

ED-07 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance  in  language 
arts  and  mathematics  in  high-poverty  elementary  schools. 

Curriculum  or  Instructional 
Practice 

ED-09 

To  design  and  implement  curricula  for  use  in  after-school  programs. 

Curriculum  or  Instructional 
Practice 

ED-11 

To  assess  the  effectiveness  of  innovative  practices  including  interventions,  strategies,  and  policies  for  infants, 
toddlers,  children,  or  youth  with  disabilities. 

Curriculum  or  Instructional 
Practice 

NIH-01 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  that  frequently  emerge  in  the 
attainment  of  mathematical  proficiency. 

Curriculum  or  Instructional 
Practice 

NIH-02 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  in  the  attaining  of  proficiency  in 
scientific  domains. 

Curriculum  or  Instructional 
Practice 

NIH-02 

To  support  studies  that  can  inform  the  design  of  evidence-based,  instructional  interventions. 

Curriculum  or  Instructional 
Practice 

NSF-02 

To  investigate  the  effectiveness  of  interventions  designed  to  improve  student  learning  and  achievement  in  pre-K- 
12  science  and  mathematics  with  an  emphasis  on  middle  and  high  school. 

Curriculum  or  Instructional 
Practice 

NSF-04 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school  levels  by 
supporting  curriculum  development. 

Curriculum  or  Instructional 
Practice 

NSF-05 

To  conduct  research  on  STEM  teaching  and  learning,  create  new  learning  materials  and  teaching  strategies, 
develop  faculty  expertise,  implement  educational  innovations,  assess  learning,  and  evaluate  innovations. 

Curriculum  or  Instructional 
Practice 

NSF-05 

To  serve  faculty,  departments,  administrators,  and  education  officials  interested  in  the  measurement  of  student 
achievement  in  courses,  curricula,  programs  of  study,  and  the  cumulative  undergraduate  experience. 

Curriculum  or  Instructional 
Practice 

NSF-11 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning,  teaching 
strategies,  and  reform  policies  and  outcomes). 

Curriculum  or  Instructional 
Practice 

NSF-12 

To  encourage  studies  of  the  effectiveness  and  impact  of  instructional  materials  developed  with  NSF  support. 

Curriculum  or  Instructional 
Practice 

NSF-13 

To  contribute  to  the  understanding  of  the  processes  that  support  improvement  of  K-12  mathematics  and  science 
teaching  and  learning  using  Math  and  Science  Partnership  projects  as  research  sites. 

Curriculum  or  Instructional 
Practices 

NSF-11 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching,  policies  of 
educational  reform,  and  outcomes  of  standards-based  reform. 

Equity:  Disabilities 

ED-12,13,14 

To  develop  or  conduct  research  on  technology -based  approaches  for  improving  access  to  and  participation  in  the 
general  curriculum  for  students  with  disabilities,  or  developmental^  appropriate  activities  for  preschool  children 

Equity:  Disabilities 

ED- 11 

To  improve  results  for  infants,  toddlers,  children,  or  youth  with  disabilities  through  early  intervention, 
educational,  transitional,  post  secondary,  or  related  services. 

Equity:  Disabilities 

ED- 11 

To  assess  the  effectiveness  of  innovative  practices  including  interventions,  strategies,  and  policies  for  infants, 
toddlers,  children,  or  youth  with  disabilities. 

Equity:  Disabilities 

ED-12,13,14 

To  develop  or  conduct  research  on  technology-based  approaches  for  improving  the  results  of  K-12  education  or 
early  intervention  for  students  with  disabilities. 

Equity:  Disabilities 

ED-12,13,14 

To  develop  or  conduct  research  on  technology -based  approaches  for  improving  accountability  and  participation 
in  statewide  assessment  and  accountability  systems  for  students  with  disabilities. 

Equity:  Disabilities 

NSF-07 

To  increase  the  participation  and  achievement  of  persons  with  disabilities  in  science,  technology,  engineering, 
and  mathematics  (STEM)  education  and  careers. 

Equity:  Disabilities 

NSF-14 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

Equity:  Gender 

NIH-01 

To  investigate  how  gender  may  moderate  the  development  of  mathematical  proficiency. 

Equity:  Gender 

NSF-06 

To  broaden  the  participation  of  girls  and  women  in  STEM  education  by  supporting  research,  dissemination  of 
research,  and  integration  of  proven  good  practices  that  will  ultimately  lead  to  a  larger  and  more  diverse  domestic 
science  and  engineering  workforce. 

Equity:  Gender 

NSF-06 

To  build  a  strong  research  base  in  gender  differences  in  STEM. 

Equity:  Gender 

NSF-06 

To  adapt  and  replicate  proven  approaches  in  new  settings  via  supplements  to  other  NSF  grants  that  have  not 
addressed  issues  related  to  gender. 

Equity:  Gender 

NSF-14 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

Equity:  Learning  Difficulties 

NIH-01 

To  delineate  the  nature  and  extent  of  specific  learning  disabilities  in  mathematics,  including  diagnosis, 
classification,  etiology,  prevention,  and  treatment. 

Equity:  Learning  Difficulties 

NIH-01 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  that  frequently  emerge  in  the 
attainment  of  mathematical  proficiency. 

Equity:  Learning  Difficulties 

NIH-02 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  in  the  attaining  of  proficiency  in 
scientific  domains. 

Equity:  Learning  Difficulties 

NIH-02 

To  find  ways  to  address  the  kinds  of  learning  difficulties  that  may  arise  in  the  attaining  of  proficiency  in  scientific 
domains. 

Equity:  Race/Ethnicity 

ED-02 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap  between 
minority  and  non-minority  students  and  between  economically  disadvantaged  students  and  their  more 
advantaged  peers. 

Equity:  Race/Ethnicity 

ED-04 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery  settings  with 
diverse  student  populations. 

Equity:  Race/Ethnicity 

ED-06 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will  result  in  closing 
achievement  gaps  between  minority  and  non-minority  students,  and  between  economically  disadvantaged 
students  and  their  more  advantaged  peers. 

Equity:  Race/Ethnicity 

NIH-01 

To  investigate  how  ethnicity  may  moderate  the  development  of  mathematical  proficiency. 

Equity:  Race/Ethnicity 

NSF-14 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

Equity:  SES 

ED-01 

Focus  is  on  preschools  that  serve  children  from  low  income  backgrounds 

Equity:  SES 

ED-02 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap  between 
minority  and  non-minority  students  and  between  economically  disadvantaged  students  and  their  more 
advantaged  peers. 

Equity:  SES 

ED-04 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery  settings  with 
diverse  student  populations. 

Equity:  SES 

ED-06 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will  result  in  closing 
achievement  gaps  between  minority  and  non-minority  students,  and  between  economically  disadvantaged 
students  and  their  more  advantaged  peers. 

Equity:  SES 

ED-07 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance  in  language 
arts  and  mathematics  in  high-poverty  elementary  schools. 

Equity:  SES 

NIH-01 

To  study  the  effects  of  poverty  on  the  failure  to  develop  mathematical  proficiency,  and  the  identification  of  risk 
and  protective  factors  within  these  contexts. 

Evaluation  &  Assessment 

ED-01 

To  implement  rigorous  evaluations  of  preschool  curricula  that  will  provide  information  to  support  informed 
choices  of  classroom  curricula  for  early  childhood  programs. 

Evaluation  &  Assessment 

ED-01 

To  implement  rigorous  evaluations  of  preschool  curricula  that  will  provide  information  to  support  informed 
choices  of  classroom  curricula  for  early  childhood  programs. 

Evaluation  &  Assessment 

ED-01 

To  support  evaluations  of  curricula  that  focus  on  child  outcomes,  use  instructional  approaches  supported  by 
scientific  literature,  and  have  standardized  training  procedures  and  materials  to  support  implementation. 

Evaluation  &  Assessment 

ED-02 

To  validate  new  or  existing  assessments  of  teacher  quality  for  teachers  of  reading/writing,  mathematics,  or 
science  at  any  grade  level  from  pre-kindergarten  through  grade  12  against  measures  of  student  achievement. 

Evaluation  &  Assessment 

ED-02 

To  establish  the  efficacy  of  existing  professional  development  programs  for  teaching  reading  or  writing  or 
mathematics  or  science  from  pre-kindergarten  through  the  middle  school  grades  with  small-scale  efficacy  or 
replication  trials. 

Evaluation  &  Assessment 

ED-02 

To  provide  evidence  of  the  effectiveness  of  teacher  preparation  or  professional  development  programs  for 
teachers  of  reading/writing,  mathematics,  or  science  from  pre-kindergarten  through  grade  12  that  are  taken  to 
scale. 

Evaluation  &  Assessment 

ED-04 

To  determine  if  programs  implemented  at  a  distance  from  the  developers  of  the  programs  and  with  no  more 
support  from  the  developers  of  the  programs  that  would  be  available  under  normal  conditions  are  effective  in  a 
variety  of  settings. 

Evaluation  &  Assessment 

ED-04 

To  support  scientific  research  that  identifies  conditions  under  which  effective  evidence -based  interventions  (i.e., 
interventions  which  have  been  shown  through  randomized  field  trials  or  well-designed  quasi-experimental 
evaluations  to  improve  student  learning  and  achievement)  succeed  when  applied  on  a  large  scale. 

Evaluation  &  Assessment 

ED-04 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery  settings  with 
diverse  student  populations. 

Evaluation  &  Assessment 

ED-06 

To  establish  the  efficacy  of  existing  interventions  and  approaches  to  mathematics  and  science  education  with 
small  efficacy  or  replication  trials. 

Evaluation  &  Assessment 

ED-06 

To  provide  evidence  on  the  effectiveness  of  mathematics  and  science  interventions  taken  to  scale. 

Evaluation  &  Assessment 

ED-07 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance  in  language 
arts  and  mathematics  in  high-poverty  elementary  schools. 

Evaluation  &  Assessment 

ED-08 

To  establish  the  efficacy  of  existing  education  interventions  that  are  used  in  schools  and  other  education  delivery 
settings. 

Evaluation  &  Assessment 

ED-08 

To  provide  federal  support  for  evaluations  of  the  effectiveness  of  education  interventions  that  are  being  used  in 
the  field,  that  appear  promising  based  on  student  performance  or  fill  an  unmet  need,  but  that  have  not  benefited 
from  a  rigorous  evaluation  of  effectiveness. 

Evaluation  &  Assessment 

ED-09 

To  evaluate  curricula  for  use  in  after-school  programs. 

Evaluation  &  Assessment 

ED- 10 

To  assess  the  impact  of  using  educational  technologies  that  are  intended  to  improve  student  academic 
achievement  in  reading  and/or  mathematics. 

Evaluation  &  Assessment 

ED- 11 

To  determine  if  the  model  is  effective  when  implemented  at  a  distance  from  the  developers  of  the  program  and 
with  no  more  support  from  the  developers  of  the  program  than  would  be  available  under  typical  conditions. 

Evaluation  &  Assessment 

ED- 11 

To  determine  the  degree  to  which  these  models  are  effective  when  implemented  by  typical  service  providers  in 
typical  settings. 

Evaluation  &  Assessment 

ED- 11 

To  assess  the  effectiveness  of  a  proven  model  or  practice  when  systematically  replicated  across  a  variety  of 
settings  by  typical  service  providers. 

Evaluation  &  Assessment 

NSF-01 

To  seek  proposals  that  offer  unique  approaches  to  evaluation  practice  in  the  generation  of  knowledge  for  the 

STEM  education  community. 

Evaluation  &  Assessment 

NSF-01 

To  build  a  strong  research  base  in  evaluation. 

Evaluation  &  Assessment 

NSF-02 

To  develop  and  document  the  psychometric  properties  of  test  items  that  are  designed  to  measure  learning  critical 
to  scaling  up  research,  e.g.,  to  develop  measures  that  assess  the  fidelity  of  implementations,  student  knowledge, 
teacher  knowledge,  or  other  important  predictor  or  outcome  variables  related  to  scale-up.  The  study  of  measures 
that  use  technology  as  an  essential  component  is  especially  encouraged,  e.g.,  a  variety  of  concept  inventories  are 
currently  available  in  several  science  disciplines;  studies  of  their  reliability  and  validity  are  important  before  they 
can  be  used  widely  for  scaling  up  research. 

Evaluation  &  Assessment 

NSF-05 

To  develop  new  assessment  materials  (tools)  and  processes  for  use  in  single  or  multiple  undergraduate 
disciplines. 

Evaluation  &  Assessment 

NSF-12 

Research  can  range  from  meta-analyses  of  existing  students  to  large-scale  studies  involving  the  design  and 
administration  of  new  evaluation  tools. 

Evaluation  &  Assessment 

NSF-13 

To  develop  new  models  and  tools  for  documenting  Math  and  Science  Partnership  projects'  progress  toward  their 
goals. 

Faculty  Development 

NSF-04 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school  levels  by 
internships  and  field  experiences  for  faculty,  teachers,  and  students;  and  other  activities. 

Faculty  Development 

NSF-04 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school  levels  by 
supporting  the  preparation  and  professional  development  of  college  faculty. 

Faculty  Development 

NSF-07 

To  support  research  on  graduate  education 

Faculty  Development 

NSF-09 

To  discover  successful  strategies  useful  to  educators  for  promoting  broader  adoption  or  adaptation  of  the 
factors(s)  related  to  students  receiving  associate,  baccalaureate,  of  graduate  degrees  in  STEM  fields  of  study. 

Faculty  Development 

NSF-09 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating  associate  and/or 
baccalaureate  degree  attainment  in  STEM. 

Faculty  Development 

NSF-09 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating  persistence  to 

STEM  graduate  study. 

Faculty  Development 

NSF-11 

To  increase  and  diversify  the  cadre  of  national  leaders  of  K-12  STEM  education  through  innovative  programs  for 
doctoral  and  postdoctoral  students. 

Faculty  Development 

NSF-11 

To  have  some  faculty  research  focus  on  a  significant  national  question. 

Faculty  Development 

NSF-13 

Projects  must  address  one  of  the  MSP  Key  Features:  Teacher  quantity,  quality  and  diversity;  challenging  courses 
and  curricula;  institutional  change  and  sustainability  (. .  .to  ensure  coordinated  institutional  change  at  the  college 
and  university  and  at  the  K-12  levels);  partnership-driven,  and  evidence  based  design  and  outcomes." 

Faculty  Development 

NSF-14 

To  provide  innovative  educational,  research,  and  career  development  opportunities  for  all  center  participants. 

Schoolwide  &  Systemwide 
Reforms 

ED-03 

To  contribute  to  building  a  foundation  for  reform  based  on  long-term,  in-class  research  that  speaks  to  reform 
goals,  teacher  professional  development  needs,  and  everyday  administrative  contexts. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  track  the  implementation  of  several  improvement  efforts  (Accelerated  Schools,  American  Choice,  and 

Success  for  All)  in  schools,  and  investigating  the  impact  on  teachers,  students,  and  schools. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance  in  language 
arts  and  mathematics  in  high-poverty  elementary  schools. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance  in  language 
arts  and  mathematics  in  high-poverty  elementary  schools. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  examine  how  state  and  local  policies  assist  or  detract  from  school  improvement  initiatives. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  gain  a  deeper  understanding  of  the  processes  of  school  improvement. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  study  how  reforms  —  in  policy,  organization,  or  structure  —  lead  to  improvements  in  instruction. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  study  how  knowledge  and  experience  influence  reforms. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching  and  student 
learning  in  language  arts  and  mathematics. 

Schoolwide  &  Systemwide 
Reforms 

ED-07 

To  study  how  the  links  between  knowledge,  reforms  and  practice  can  be  strengthened. 

Schoolwide  &  Systemwide 
Reforms 

ED-12,13,14 

To  develop  or  conduct  research  on  technology-based  approaches  for  improving  accountability  and  participation 
in  statewide  assessment  and  accountability  systems  for  students  with  disabilities. 

Schoolwide  &  Systemwide 
Reforms 

NSF-02 

To  identify  the  conditions  under  which  effective,  evidence-based  interventions  to  improve  preK-12  student 
learning  and  achievement  succeed  when  applied  on  a  large  scale. 

Schoolwide  &  Systemwide 
Reforms 

NSF-08 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  and  to  STEM 
education  with  research  across  a  continuum  that  includes  the  diffusion  of  STEM  innovations. 

Schoolwide  &  Systemwide 
Reforms 

NSF-11 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching,  policies  of 
educational  reform,  and  outcomes  of  standards-based  reform. 

Schoolwide  &  Systemwide 
Reforms 

NSF-14 

To  understand  what  learning  is  and  how  it  is  affected  at  all  levels,  ranging  from  the  digital  to  the  societal. 

STEM  Education  Policies 

ED-07 

To  examine  how  state  and  local  policies  assist  or  detract  from  school  improvement  initiatives. 

STEM  Education  Policies 

NSF-01 

To  seek  proposals  that  offer  unique  approaches  for  broad  policymaking  within  the  research  and  education 
enterprise. 

STEM  Education  Policies 

NSF-02 

To  provide  State  and  local  policy  makers,  as  well  as  school-level  administrators  and  university  faculty  and 
administrators,  with  information  on  efforts  at  improvement  that  have  led  to  increased  and  sustained  student 
learning. 

STEM  Education  Policies 

NSF-08 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  and  to  STEM 
education  with  research  across  a  continuum  that  include  STEM  policy  research. 

STEM  Education  Policies 

NSF-11 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning,  teaching 
strategies,  and  reform  policies  and  outcomes) 

STEM  Education  Policies 

NSF-11 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching,  policies  of 
educational  reform,  and  outcomes  of  standards-based  reform. 

STEM  Workforce 

Improvement 

NSF-04 

To  strengthen  the  education  of  technicians  for  careers  in  biotechnology,  environmental  technology,  information 
technology,  manufacturing  and  many  other  science-and  engineering-related  fields  that  drive  our  nation’s 
economy. 

STEM  Workforce 

Improvement 

NSF-06 

To  contribute  to  the  knowledge  base  addressing  gender-related  differences  in  learning  and  in  the  educational 
experiences  that  affect  student  interest,  performance,  and  choice  of  careers. 

STEM  Workforce 

Improvement 

NSF-06 

To  broaden  the  participation  of  girls  and  women  in  STEM  education  by  supporting  research,  dissemination  of 
research,  and  integration  of  proven  good  practices  that  will  ultimately  lead  to  a  larger  and  more  diverse  domestic 
science  and  engineering  workforce. 

STEM  Workforce 

Improvement 

NSF-09 

To  increase  the  number  of  students  (U.S.  citizens  or  permanent  residents)  receiving  associate  or  baccalaureate 
degrees  in  established  or  emerging  fields  within  science,  technology,  engineering,  and  mathematics  (STEM). 

STEM  Workforce 

Improvement 

NSF-09 

To  support  educational  research  projects  of  associate  or  baccalaureate  degree  attainment  in  STEM. 

STEM  Workforce 

Improvement 

NSF-09 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating  undergraduate 
access  to  STEM  careers. 

STEM  Workforce 

Improvement 

NSF-14 

To  incorporate  an  integrated,  multidisciplinary  research  program  focused  on  a  clear  intellectual  core  and 
connected  to  specific  scientific,  technological,  educational  and/or  workforce  challenges. 

Student  Cognition  &  Learning 

ED-01 

Outcomes  of  interest  include  language  development,  pre-reading  and  pre-mathematics  abilities,  cognition, 
general  knowledge  and  social  competence. 

Student  Cognition  &  Learning 

ED-05 

To  address  basic  or  higher-order  cognitive  processes  and  directly  link  those  processes  to  improving  student 
learning  and  achievement. 

Student  Cognition  &  Learning 

ED-05 

To  apply  recent  theoretical  and  empirical  advances  in  understanding  learning  from  cognitive  science,  cognitive 
psychology,  and  neuroscience  research  to  education  practice  with  the  goal  of  improving  student  learning  and 
academic  achievement. 

Student  Cognition  &  Learning 

ED-05 

To  better  understand  learning  and  cognitive  processing  as  it  occurs  in  the  classroom  -  a  cognitively  rich 
environment  in  which  multiple  activities  occur  simultaneously  -  so  that  instructional  approaches  can  be 
developed  that  maximize  student  learning. 

Student  Cognition  &  Learning 

ED-05 

To  develop  instruction  practice  or  materials  based  on  general  principles  of  learning  and  information  processing 
gained  from  cognitive  science  and  test  the  effects  of  these  new  approaches  within  education  delivery  settings. 

Student  Cognition  &  Learning 

ED-05 

To  enhance  our  understanding  of  the  practical  challenges  of  using  the  findings  of  cognitive  science  to  transform 
the  practice  of  teaching  and  learning. 

Student  Cognition  &  Learning 

ED-05 

To  establish  a  scientific  foundation  for  education  by  building  on  recent  theoretical  and  empirical  advances  in 
understanding  learning  from  cognitive  science,  cognitive  psychology,  and  neuroscience  research. 

Student  Cognition  &  Learning 

ED-05 

To  improve  student  learning  by  bringing  recent  advances  in  cognitive  science  to  bear  on  significant  problems  in 
education. 

Student  Cognition  &  Learning 

ED-05 

To  support  cognitive  science  research  that  will  be  conducted  primarily  in  education  delivery  settings. 

Student  Cognition  &  Learning 

ED-05 

To  support  research  on  key  processes  of  attention,  memory,  and  reasoning  that  are  essential  for  learning  and  that 
are  likely  to  produce  substantial  gains  in  academic  achievement. 

Student  Cognition  &  Learning 

ED-07 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching  and  student 
learning  in  language  arts  and  mathematics. 

Student  Cognition  &  Learning 

ED- 10 

To  assess  the  impact  of  using  educational  technologies  that  are  intended  to  improve  student  academic 
achievement  in  reading  and/or  mathematics. 

Student  Cognition  &  Learning 

NIH-01 

To  delineate  skill  sets  needed  to  attain  mathematical  proficiency. 

Student  Cognition  &  Learning 

NIH-01 

To  encourage  basic  and  intervention  research  in  all  aspects  of  mathematical  thinking  and  problem  solving. 

Student  Cognition  &  Learning 

NIH-01 

To  investigate  individual  differences  that  may  moderate  achievement  in  mathematics. 

Student  Cognition  &  Learning 

NIH-01 

To  study  the  effects  of  poverty  on  the  failure  to  develop  mathematical  proficiency,  and  the  identification  of  risk 
and  protective  factors  within  these  contexts. 

Student  Cognition  &  Learning 

NIH-01 

To  support  research  on  the  normal  development  of  mathematical  proficiency,  including  both  conceptual  and 
procedural  knowledge.  Specific  domains  of  interest  include,  but  are  not  limited  to:  basic  numerical 
representations  and  processing,  arithmetic  comprehension  and  procedural  skills,  proficiency  with  fractions  and 
other  types  of  rational  numbers,  algebraic  problem  solving,  geometric  thinking,  concepts  of  probability  and 
chance,  and  measurement  concepts  and  skills. 

Student  Cognition  &  Learning 

NIH-01 

To  support  studies  that  explore  a  variety  of  influences  on  normal  and  atypical  development  in  mathematical 
learning  and  cognition,  including  genetic  and  neurobiological  substrates,  as  well  as  cognitive,  linguistic, 
sociocultural,  and  instructional  factors. 

Student  Cognition  &  Learning 

NIH-02 

To  delineate  skill  sets  needed  to  attain  proficiency  in  scientific  domains. 

Student  Cognition  &  Learning 

NIH-02 

To  encourage  basic  and  intervention  research  in  scientific  reasoning,  learning,  and  discovery. 

Student  Cognition  &  Learning 

NIH-02 

To  encourage  research  on  causal  thinking  and  inference,  theory-evidence  coordination,  and  reasoning  about  data. 

Student  Cognition  &  Learning 

NIH-02 

To  encourage  research  on  factors  contributing  to  conceptual  change,  as  are  studies  of  inductive  and  deductive 
reasoning,  and  the  acquisition  of  scientific  concepts  such  as  experimental  control  and  falsifiability. 

Student  Cognition  &  Learning 

NIH-02 

To  improve  our  understanding  of  the  cognitive  and  developmental  bases  of  scientific  thinking  and  learning. 

Student  Cognition  &  Learning 

NIH-02 

To  investigate  developmental  changes  in  naive  thinking  about  the  biological  and  physical  worlds. 

Student  Cognition  &  Learning 

NIH-02 

To  investigate  individual  differences  that  may  moderate  achievement  in  science. 

Student  Cognition  &  Learning 

NIH-02 

To  support  studies  that  explore  a  variety  of  influences  on  normal  and  atypical  development  in  science  learning 
and  cognition,  including  neurobiological  substrates,  as  well  as  cognitive,  linguistic,  sociocultural,  and 
instructional  factors. 

Student  Cognition  &  Learning 

NSF-02 

To  focus  on  mathematics  and  science  at  the  middle  and  high  school  levels. 

Student  Cognition  &  Learning 

NSF-02 

To  investigate  the  effectiveness  of  interventions  designed  to  improve  student  learning  and  achievement  in  pre-K- 
12  science  and  mathematics  with  an  emphasis  on  middle  and  high  school. 

Student  Cognition  &  Learning 

NSF-03 

To  advance  a  rigorous  understanding  of  how  the  human  brain  supports  thought,  perception,  affect,  action,  social 
processes,  and  other  aspects  of  cognition  and  behavior.  Topics  may  bear  on  core  functions  such  as  sensory, 
learning,  language,  reasoning,  emotion,  and  executive  processes,  or  more  specialized  processes  such  as  empathy, 
creativity,  representation  of  self  and  other,  or  intentionality,  among  many  other  possibilities.  Topics  may  also 
include  how  such  processes  develop  and  change  in  the  brain.  (Cognitive  Neuroscience  Program) 

Student  Cognition  &  Learning 

NSF-03 

To  support  studies  that  increase  our  understanding  of  cognitive,  linguistic,  social,  cultural,  and  biological 
processes  related  to  children's  and  adolescents'  development.  (Developmental  and  Learning  Sciences  Program). 

Student  Cognition  &  Learning 

NSF-04 

To  improve  technician  education  with  an  emphasis  on  two-year  colleges. 

Student  Cognition  &  Learning 

NSF-05 

To  serve  faculty,  departments,  administrators,  and  education  officials  interested  in  the  measurement  of  student 
achievement  in  courses,  curricula,  programs  of  study,  and  the  cumulative  undergraduate  experience. 

Student  Cognition  &  Learning 

NSF-07 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  in  science, 
technology,  engineering,  and  mathematics  (STEM)  education. 

Student  Cognition  &  Learning 

NSF-08 

To  support  research  across  a  continuum  that  includes  behavioral,  cognitive,  affective  and  social  aspects  of  human 
learning. 

Student  Cognition  &  Learning 

NSF-08 

To  support  research  across  a  continuum  that  includes  changing  educational  systems  to  improve  STEM  learning. 

Student  Cognition  &  Learning 

NSF-11 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning,  teaching 
strategies,  and  reform  policies  and  outcomes). 

Student  Cognition  &  Learning 

NSF-11 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching,  policies  of 
educational  reform,  and  outcomes  of  standards-based  reform. 

Student  Cognition  &  Learning 

NSF-12 

To  support  studies  that  examine  the  effectiveness  and  impact  of  NSF  funded  efforts  to  enhance  teachers'  and 
students'  STEM  learning. 

Student  Cognition  &  Learning 

NSF-13 

To  contribute  to  the  understanding  of  the  processes  that  support  improvement  of  K-12  mathematics  and  science 
teaching  and  learning  using  Math  and  Science  Partnership  projects  as  research  sites. 

Student  Cognition  &  Learning 

NSF-14 

To  advance  the  frontiers  of  all  the  sciences  of  learning  through  integrated  research. 

Student  Cognition  &  Learning 

NSF-14 

To  understand  what  learning  is  and  how  it  is  affected  at  all  levels,  ranging  from  the  digital  to  the  societal. 

Teacher  Development 

ED-02 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap  between 
minority  and  non-minority  students  and  between  economically  disadvantaged  students  and  their  more 
advantaged  peers. 

Teacher  Development 

ED-02 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  substantial  gains  in  academic  achievements  for  all 
students. 

Teacher  Development 

ED-02 

To  identify  effective  strategies  for  improving  the  performance  of  classroom  teachers  in  ways  that  increase 
student  learning  and  school  achievement. 

Teacher  Development 

ED-02 

To  support  the  development  of  new  professional  development  programs  for  teaching  reading  or  writing  skills  or 
mathematics  or  science  from  pre-Kindergarten  through  the  middle  school  grades  that  will  eventually  result  in 
improving  teacher  practices  and  through  them  student  achievement. 

Teacher  Development 

ED-03 

To  contribute  to  building  a  foundation  for  reform  based  on  long-term,  in-class  research  that  speaks  to  reform 
goals,  teacher  professional  development  needs,  and  everyday  administrative  contexts. 

Teacher  Development 

NSF-04 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school  levels  by 
internships  and  field  experiences  for  faculty,  teachers,  and  students;  and  other  activities. 

Teacher  Development 

NSF-04 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school  levels  by 
supporting  the  preparation  and  professional  development  of  secondary  school  teachers. 

Teacher  Development 

NSF-06 

To  contribute  to  the  knowledge  base  on  how  pedagogical  approaches  and  teaching  styles,  curriculum,  student 
services,  and  institutional  culture  contribute  to  causing  or  closing  gender  gaps  that  persist  in  certain  fields. 

Teacher  Development 

NSF-06 

To  adapt  and  replicate  proven  approaches  in  new  settings  via  supplements  to  other  NSF  grants  that  have  not 
addressed  issues  related  to  gender. 

Teacher  Development 

NSF-10 

To  address  critical  issues  and  needs  regarding  the  recruitment,  preparation,  induction,  retention,  and  life-long 
development  of  K-12  science,  technology,  engineering,  and  mathematics  (STEM)  teachers. 

Teacher  Development 

NSF-10 

To  improve  the  quality  and  coherence  of  teacher  learning  experiences  across  the  continuum  through  research  that 
informs  teaching  practice  and  the  development  of  innovative  resources  for  the  professional  development  of  K-12 
STEM  teachers. 

Teacher  Development 

NSF-10 

To  promote  the  quality  of  K-12  teachers  in  a  coherent  way  throughout  teachers'  careers  with  work  that 
acknowledges  the  continuous  process  of  teachers  learning  rather  than  as  a  set  of  unrelated  efforts. 

Teacher  Development 

NSF-10 

To  synthesize  and  further  advance  a  compelling  body  of  research  that  will  both  inform  and  strengthen  STEM 
teacher  effectiveness  and  classroom  instruction. 

Teacher  Development 

NSF-11 

To  increase  the  number  ofK-16  educators  capable  of  delivering  high-quality  STEM  instruction  and  assessment 

Teacher  Development 

NSF-11 

To  focus  on  the  advanced  preparation  of  science,  technology,  engineering,  and  mathematics  (STEM)  educators, 
as  well  as  the  establishment  of  meaningful  partnerships  among  education  stakeholders,  especially  Ph.D. -granting 
institutions,  school  systems,  and  informal  education  performers. 

Teacher  Development 

NSF-11 

To  renew  and  diversify  the  cadre  of  educators  in  STEM  education  in  both  formal  and  informal  settings. 

Teacher  Development 

NSF-12 

To  support  studies  that  examine  the  effectiveness  and  impact  of  NSF  funded  efforts  to  enhance  teachers'  and 
students'  STEM  learning 

Teacher  Development 

NSF-13 

Projects  must  address  one  of  the  MSP  Key  Features:  Teacher  quantity,  quality  and  diversity;  challenging  courses 
and  curricula;  institutional  change  and  sustainability  (. .  .to  ensure  coordinated  institutional  change  at  the  college 
and  university  and  at  the  K-12  levels);  partnership-driven,  and  evidence  based  design  and  outcomes." 

70 


US  Department  of  Education 

ED- 1  Preschool  Curriculum  Evaluation  Research  (PCER)  Program 
ED-2  Teacher  Quality  Research  Grants  (TQR) 

ED-3  National  Center  for  Improving  Student  Learning  &  Achievement  in  Math  &  Science(closed  2/2004) 
ED-4  Interagency  Education  Research  Initiative  (IERI) 

ED-5  Cognition  &  Student  Learning  (CASL)  Research  Grants 
ED-6  Mathematics  &  Science  Education  Research  Grants 

ED-7  Consortium  for  Policy  Research  in  Education  Study  of  Instructional  Improvement 
ED-8  Field  Initiated  Evaluations  of  Education  Interventions  (formerly  Field  Initiated  Studies) 

ED-9  Development,  Implementation  &  Eval  of  Academic  Instruction  for  After-School  Prog 

ED- 10  Evaluation  of  the  Effectiveness  of  Educational  Technology  Intervention 

ED- 1 1  Research  and  Innovation 

ED- 12  Steppingstones  of  Technology  Innovation 

ED- 13  Research  Institutes  on  Technology 

ED- 14  Research  on  Educational  Captioning 

ED- 15  Pre-Doctoral  Interdisciplinary  Research  Training  Fellowship  Program 
ED- 16  Post-Doctoral  Research  Training  Program 


National  Institutes  of  Health 

NIH-1  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Math 

NIH-2  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Science 

National  Science  Foundation 

NSF-1  Evaluative  Research  &  Evaluation  Capacity  Building  (EREC) 

NSF-2  Interagency  Education  Research  Initiative 

NSF-3  Various  programs  in  Social  Behavioral  and  Economic  Sciences  (SBE) 

NSF-4  Advanced  Technological  Education  (ATE) 

NSF-5  Course,  Curriculum  &  Laboratory  Improvement/Assessment  of  Student  Achievement 
NSF-6  Research  on  Gender  in  Science  &  Engineering  (GSE) 

NSF-7  Research  Disabilities  Education  (RDE) 

NSF-8  Research  on  Learning  &  Education  (ROLE) 

NSF-9  Science,  Technology,  Engineering  and  Mathematics  Talent  Expansion  Program  (STEP) 
NSF-10  Program  Teacher  Professional  Continuum  (TPC) 

NSF-1 1  Centers  for  Learning  and  Teaching 

NSF-1 2  Instructional  Materials  Development  (IMD)- Applied  Research  Component 

NSF-1 3  Research,  Evaluation  and  Technical  Assistance  (RET A) 

NSF-14  Science  of  Learning  Centers  (SLC) 
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STEM  Education  Research  -  Appendix  D 

Appendix  D:  Specific  Objectives  of  Federal  STEM  Education  Research  Programs  --  Sorted  By  Program 


Specific  Objectives  of  Federal  STEM  Education  Research  Programs  —  Sorted  By  Program 


Program  Category  Program  Objectives 

(derived  from  published  program  authority4) 


ED-01 

Curriculum  or  Instructional  Practice 

To  conduct  complementary  research  studies  and  a  cross-site  program  evaluation  including  studies 
that  address  how  individual  or  background  differences  in  children  interact  with  the  curriculum  to 
influence  developmental  outcomes 

ED-01 

Curriculum  or  Instructional  Practice 

To  conduct  complementary  research  studies  and  a  cross-site  program  evaluation  including  studies 
that  compare  different  versions  of  the  curriculum  or  different  approaches  to  implementation  in  order 
to  identify  key  features  of  the  curriculum  and  approaches  that  might  improve  effectiveness  and  ease 
of  implementation. 

ED-01 

Curriculum  or  Instructional  Practice 

To  determine  whether  one  or  more  curricula  produce  educationally  meaningful  effects  on  children. 

ED-01 

Focus  is  on  preschools  that  serve  children  from  low  income  backgrounds 

ED-01 

Evaluation  &  Assessment 

To  implement  rigorous  evaluations  of  preschool  curricula  that  will  provide  information  to  support 
informed  choices  of  classroom  curricula  for  early  childhood  programs. 

ED-01 

Evaluation  &  Assessment 

To  implement  rigorous  evaluations  of  preschool  curricula  that  will  provide  information  to  support 
informed  choices  of  classroom  curricula  for  early  childhood  programs. 

ED-01 

Evaluation  &  Assessment 

To  support  evaluations  of  curricula  that  focus  on  child  outcomes,  use  instructional  approaches 
supported  by  scientific  literature,  and  have  standardized  training  procedures  and  materials  to  support 
implementation. 

ED-02 

Student  Cognition  &  Learning 

Outcomes  of  interest  include  language  development,  pre-reading  and  pre-mathematics  abilities, 
cognition,  general  knowledge  and  social  competence. 

ED-02 

Equity:  Race/Ethnicity 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap 
between  minority  and  non-minority  students  and  between  economically  disadvantaged  students  and 
their  more  advantaged  peers. 

ED-02 

Equity:  SES 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap 
between  minority  and  non-minority  students  and  between  economically  disadvantaged  students  and 
their  more  advantaged  peers. 

ED-02 

Evaluation  &  Assessment 

To  validate  new  or  existing  assessments  of  teacher  quality  for  teachers  of  reading/writing, 
mathematics,  or  science  at  any  grade  level  from  pre -kindergarten  through  grade  12  against  measures 
of  student  achievement. 

4  Program  authority  includes  Requests  for  Proposals/ Applications,  Program  Announcements,  other  types  of  grant-related  solicitations,  legislative  language,  and  web-based  program 
descriptions. 


ED-02 

Evaluation  &  Assessment 

To  establish  the  efficacy  of  existing  professional  development  programs  for  teaching  reading  or 
writing  or  mathematics  or  science  from  pre -kindergarten  through  the  middle  school  grades  with 
small-scale  efficacy  or  replication  trials. 

ED-02 

Evaluation  &  Assessment 

To  provide  evidence  of  the  effectiveness  of  teacher  preparation  or  professional  development 
programs  for  teachers  of  reading/ writing,  mathematics,  or  science  from  pre-kindergarten  through 
grade  12  that  are  taken  to  scale. 

ED-02 

Teacher  Development 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  a  reduction  of  the  achievement  gap 
between  minority  and  non-minority  students  and  between  economically  disadvantaged  students  and 
their  more  advantaged  peers. 

ED-02 

Teacher  Development 

To  address  issues  of  teacher  quality  that  are  likely  to  lead  to  substantial  gains  in  academic 
achievements  for  all  students. 

ED-02 

Teacher  Development 

To  identify  effective  strategies  for  improving  the  performance  of  classroom  teachers  in  ways  that 
increase  student  learning  and  school  achievement. 

ED-03 

Teacher  Development 

To  support  the  development  of  new  professional  development  programs  for  teaching  reading  or 
writing  skills  or  mathematics  or  science  from  pre-Kindergarten  through  the  middle  school  grades 
that  will  eventually  result  in  improving  teacher  practices  and  through  them  student  achievement. 

ED-03 

Curriculum  or  Instructional  Practice 

To  construct  learning  environments  that  exemplify  current  research  and  theory  about  effective 
learning  of  math  and  science. 

ED-03 

Schoolwide  &  Systemwide  Reforms 

To  contribute  to  building  a  foundation  for  reform  based  on  long-term,  in-class  research  that  speaks 
to  reform  goals,  teacher  professional  development  needs,  and  everyday  administrative  contexts. 

ED-04 

Teacher  Development 

To  contribute  to  building  a  foundation  for  reform  based  on  long-term,  in-class  research  that  speaks 
to  reform  goals,  teacher  professional  development  needs,  and  everyday  administrative  contexts. 

ED-04 

Equity:  Race/Ethnicity 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery 
settings  with  diverse  student  populations. 

ED-04 

Equity:  SES 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery 
settings  with  diverse  student  populations. 

ED-04 

Evaluation  &  Assessment 

To  determine  if  programs  implemented  at  a  distance  from  the  developers  of  the  programs  and  with 
no  more  support  from  the  developers  of  the  programs  that  would  be  available  under  normal 
conditions  are  effective  in  a  variety  of  settings. 

ED-04 

Evaluation  &  Assessment 

To  support  scientific  research  that  identifies  conditions  under  which  effective  evidence -based 
interventions  (i.e.,  interventions  which  have  been  shown  through  randomized  field  trials  or  well- 
designed  quasi-experimental  evaluations  to  improve  student  learning  and  achievement)  succeed 
when  applied  on  a  large  scale. 

ED-05 

Evaluation  &  Assessment 

To  support  scientific  research  that  investigates  the  effectiveness  of  education  intervention  in  reading, 
mathematics,  and  the  sciences  as  that  are  implemented  in  varied  school  and  education  delivery 
settings  with  diverse  student  populations. 

ED-05 

Curriculum  or  Instructional  Practice 

To  develop  instruction  practice  or  materials  based  on  general  principles  of  learning  and  information 
processing  gained  from  cognitive  science  and  test  the  effects  of  these  new  approaches  within 
education  delivery  settings. 

ED-05 

Student  Cognition  &  Learning 

To  address  basic  or  higher-order  cognitive  processes  and  directly  link  those  processes  to  improving 
student  learning  and  achievement. 

ED-05 

Student  Cognition  &  Learning 

To  apply  recent  theoretical  and  empirical  advances  in  understanding  learning  from  cognitive 
science,  cognitive  psychology,  and  neuroscience  research  to  education  practice  with  the  goal  of 
improving  student  learning  and  academic  achievement. 

ED-05 

Student  Cognition  &  Learning 

To  better  understand  learning  and  cognitive  processing  as  it  occurs  in  the  classroom  -  a  cognitively 
rich  environment  in  which  multiple  activities  occur  simultaneously  -  so  that  instructional  approaches 
can  be  developed  that  maximize  student  learning. 

ED-05 

Student  Cognition  &  Learning 

To  develop  instruction  practice  or  materials  based  on  general  principles  of  learning  and  information 
processing  gained  from  cognitive  science  and  test  the  effects  of  these  new  approaches  within 
education  delivery  settings. 

ED-05 

Student  Cognition  &  Learning 

To  enhance  our  understanding  of  the  practical  challenges  of  using  the  findings  of  cognitive  science 
to  transform  the  practice  of  teaching  and  learning. 

ED-05 

Student  Cognition  &  Learning 

To  establish  a  scientific  foundation  for  education  by  building  on  recent  theoretical  and  empirical 
advances  in  understanding  learning  from  cognitive  science,  cognitive  psychology,  and  neuroscience 
research. 

ED-05 

Student  Cognition  &  Learning 

To  improve  student  learning  by  bringing  recent  advances  in  cognitive  science  to  bear  on  significant 
problems  in  education. 

ED-05 

Student  Cognition  &  Learning 

To  support  cognitive  science  research  that  will  be  conducted  primarily  in  education  delivery  settings. 

ED-06 

Student  Cognition  &  Learning 

To  support  research  on  key  processes  of  attention,  memory,  and  reasoning  that  are  essential  for 
learning  and  that  are  likely  to  produce  substantial  gains  in  academic  achievement. 

ED-06 

Curriculum  or  Instructional  Practice 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will 
result  in  improving  mathematics  achievement  for  all  students. 

ED-06 

Curriculum  or  Instructional  Practice 

To  support  the  development  of  new  interventions  and  approaches  to  mathematics  and  science 
education  that  will  eventually  result  in  improving  mathematics  and  science  achievement. 

ED-06 

Equity:  Race/Ethnicity 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will 
result  in  closing  achievement  gaps  between  minority  and  non-minority  students,  and  between 
economically  disadvantaged  students  and  their  more  advantaged  peers. 

ED-06 

Equity:  SES 

To  support  the  identification  of  interventions  and  approaches  in  mathematics  education  that  will 
result  in  closing  achievement  gaps  between  minority  and  non-minority  students,  and  between 
economically  disadvantaged  students  and  their  more  advantaged  peers. 

ED-06 

Evaluation  &  Assessment 

To  establish  the  efficacy  of  existing  interventions  and  approaches  to  mathematics  and  science 
education  with  small  efficacy  or  replication  trials. 

ED-07 

Evaluation  &  Assessment 

To  provide  evidence  on  the  effectiveness  of  mathematics  and  science  interventions  taken  to  scale. 

ED-07 

Curriculum  or  Instructional  Practice 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching 
and  student  learning  in  language  arts  and  mathematics. 

ED-07 

Curriculum  or  Instructional  Practice 

To  study  how  the  links  between  knowledge,  reforms  and  practice  can  be  strengthened. 

ED-07 

Curriculum  or  Instructional  Practice 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance 
in  language  arts  and  mathematics  in  high-poverty  elementary  schools. 

ED-07 

Equity:  SES 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance 
in  language  arts  and  mathematics  in  high-poverty  elementary  schools. 

ED-07 

Evaluation  &  Assessment 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance 
in  language  arts  and  mathematics  in  high-poverty  elementary  schools. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  track  the  implementation  of  several  improvement  efforts  (Accelerated  Schools,  American 

Choice,  and  Success  for  All)  in  schools,  and  investigating  the  impact  on  teachers,  students,  and 
schools. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance 
in  language  arts  and  mathematics  in  high-poverty  elementary  schools. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  understand  the  impact  of  school  improvement  programs  on  instruction  and  student  performance 

in  language  arts  and  mathematics  in  high-poverty  elementary  schools. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  examine  how  state  and  local  policies  assist  or  detract  from  school  improvement  initiatives. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  gain  a  deeper  understanding  of  the  processes  of  school  improvement. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  study  how  reforms  —  in  policy,  organization,  or  structure  —  lead  to  improvements  in  instruction. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  study  how  knowledge  and  experience  influence  reforms. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching 
and  student  learning  in  language  arts  and  mathematics. 

ED-07 

Schoolwide  &  Systemwide  Reforms 

To  study  how  the  links  between  knowledge,  reforms  and  practice  can  be  strengthened. 

ED-07 

STEM  Education  Policies 

To  examine  how  state  and  local  policies  assist  or  detract  from  school  improvement  initiatives. 

ED-08 

Student  Cognition  &  Learning 

To  investigate  the  conditions  under  which  school  improvement  efforts  improve  classroom  teaching 
and  student  learning  in  language  arts  and  mathematics. 

ED-08 

Evaluation  &  Assessment 

To  establish  the  efficacy  of  existing  education  interventions  that  are  used  in  schools  and  other 
education  delivery  settings. 

ED-09 

Evaluation  &  Assessment 

To  provide  federal  support  for  evaluations  of  the  effectiveness  of  education  interventions  that  are 
being  used  in  the  field,  that  appear  promising  based  on  student  performance  or  fill  an  unmet  need, 
but  that  have  not  benefited  from  a  rigorous  evaluation  of  effectiveness. 

ED-09 

Curriculum  or  Instructional  Practice 

To  design  and  implement  curricula  for  use  in  after-school  programs. 

ED- 10 

Evaluation  &  Assessment 

To  evaluate  curricula  for  use  in  after-school  programs. 

ED- 10 

Evaluation  &  Assessment 

To  assess  the  impact  of  using  educational  technologies  that  are  intended  to  improve  student 
academic  achievement  in  reading  and/or  mathematics. 

ED- 11 

Student  Cognition  &  Learning 

To  assess  the  impact  of  using  educational  technologies  that  are  intended  to  improve  student 
academic  achievement  in  reading  and/or  mathematics. 

ED- 11 

Capacity  Building 

To  develop,  implement,  and  evaluate  models  (including  models  for  professional  development)  for 
infants,  toddlers,  children,  or  youth  with  disabilities. 

ED- 11 

Curriculum  or  Instructional  Practice 

To  assess  the  effectiveness  of  innovative  practices  including  interventions,  strategies,  and  policies 
for  infants,  toddlers,  children,  or  youth  with  disabilities. 

ED- 11 

Equity:  Disabilities 

To  improve  results  for  infants,  toddlers,  children,  or  youth  with  disabilities  through  early 
intervention,  educational,  transitional,  post  secondary,  or  related  services. 

ED- 11 

Equity:  Disabilities 

To  assess  the  effectiveness  of  innovative  practices  including  interventions,  strategies,  and  policies 
for  infants,  toddlers,  children,  or  youth  with  disabilities. 

ED- 11 

Evaluation  &  Assessment 

To  determine  if  the  model  is  effective  when  implemented  at  a  distance  from  the  developers  of  the 
program  and  with  no  more  support  from  the  developers  of  the  program  than  would  be  available 
under  typical  conditions. 

ED- 11 

Evaluation  &  Assessment 

To  determine  the  degree  to  which  these  models  are  effective  when  implemented  by  typical  service 
providers  in  typical  settings. 

ED- 

12,13,14 

Evaluation  &  Assessment 

To  assess  the  effectiveness  of  a  proven  model  or  practice  when  systematically  replicated  across  a 
variety  of  settings  by  typical  service  providers. 

ED- 

12,13,14 

Equity:  Disabilities 

To  develop  or  conduct  research  on  technology -based  approaches  for  improving  access  to  and 
participation  in  the  general  curriculum  for  students  with  disabilities,  or  developmentally  appropriate 
activities  for  preschool  children 

ED- 

12,13,14 

Equity:  Disabilities 

To  develop  or  conduct  research  on  technology-based  approaches  for  improving  the  results  of  K-12 
education  or  early  intervention  for  students  with  disabilities. 

ED- 

12,13,14 

Equity:  Disabilities 

To  develop  or  conduct  research  on  technology-based  approaches  for  improving  accountability  and 
participation  in  statewide  assessment  and  accountability  systems  for  students  with  disabilities. 

ED- 15 

Schoolwide  &  Systemwide  Reforms 

To  develop  or  conduct  research  on  technology-based  approaches  for  improving  accountability  and 
participation  in  statewide  assessment  and  accountability  systems  for  students  with  disabilities. 

ED- 15 

Capacity  Building 

To  establish  a  network  of  training  programs  that  collectively  produce  a  cadre  of  education 
researchers  willing  and  able  to  conduct  a  new  generation  of  methodologically  rigorous  and 
educationally  relevant  scientific  research  that  will  provide  solutions  to  pressing  problems  and 
challenges  facing  American  education. 

ED- 16 

Capacity  Building 

To  support  the  development  of  innovative  interdisciplinary  training  programs  for  doctoral  students 
interested  in  conducting  applied  education  research. 

ED- 16 

Capacity  Building 

To  produce  a  cadre  of  education  researchers  willing  and  able  to  conduct  a  new  generation  of 
methodologically  rigorous  and  educationally  relevant  scientific  research  that  will  provide  solutions 
to  pressing  problems  and  challenges  facing  American  education 

ED- 16 

Capacity  Building 

To  increase  the  supply  of  scientists  and  researchers  in  education  who  are  prepared  to  conduct 
rigorous  evaluation  studies,  develop  new  products  and  approaches  that  are  grounded  in  a  science  of 
learning,  design  valid  tests  and  measures,  and  explore  data  with  sophisticated  statistical  methods. 

NIH-01 

Capacity  Building 

To  support  the  training  of  postdoctoral  fellows  interested  in  conducting  applied  education  research. 

NIH-01 

Curriculum  or  Instructional  Practice 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  that  frequently  emerge 
in  the  attainment  of  mathematical  proficiency. 

NIH-01 

Equity:  Gender 

To  investigate  how  gender  may  moderate  the  development  of  mathematical  proficiency. 

NIH-01 

Equity:  Learning  Difficulties 

To  delineate  the  nature  and  extent  of  specific  learning  disabilities  in  mathematics,  including 
diagnosis,  classification,  etiology,  prevention,  and  treatment. 

NIH-01 

Equity:  Learning  Difficulties 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  that  frequently  emerge 
in  the  attainment  of  mathematical  proficiency. 

NIH-01 

Equity:  Race/Ethnicity 

To  investigate  how  ethnicity  may  moderate  the  development  of  mathematical  proficiency. 

NIH-01 

Equity:  SES 

To  study  the  effects  of  poverty  on  the  failure  to  develop  mathematical  proficiency,  and  the 
identification  of  risk  and  protective  factors  within  these  contexts. 

NIH-01 

Student  Cognition  &  Learning 

To  delineate  skill  sets  needed  to  attain  mathematical  proficiency. 

NIH-01 

Student  Cognition  &  Learning 

To  encourage  basic  and  intervention  research  in  all  aspects  of  mathematical  thinking  and  problem 
solving. 

NIH-01 

Student  Cognition  &  Learning 

To  investigate  individual  differences  that  may  moderate  achievement  in  mathematics. 

NIH-01 

Student  Cognition  &  Learning 

To  study  the  effects  of  poverty  on  the  failure  to  develop  mathematical  proficiency,  and  the 
identification  of  risk  and  protective  factors  within  these  contexts. 

NIH-01 

Student  Cognition  &  Learning 

To  support  research  on  the  normal  development  of  mathematical  proficiency,  including  both 
conceptual  and  procedural  knowledge.  Specific  domains  of  interest  include,  but  are  not  limited  to: 
basic  numerical  representations  and  processing,  arithmetic  comprehension  and  procedural  skills, 
proficiency  with  fractions  and  other  types  of  rational  numbers,  algebraic  problem  solving,  geometric 
thinking,  concepts  of  probability  and  chance,  and  measurement  concepts  and  skills. 

NIH-02 

Student  Cognition  &  Learning 

To  support  studies  that  explore  a  variety  of  influences  on  normal  and  atypical  development  in 
mathematical  learning  and  cognition,  including  genetic  and  neurobiological  substrates,  as  well  as 
cognitive,  linguistic,  sociocultural,  and  instructional  factors. 

NIH-02 

Curriculum  or  Instructional  Practice 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  in  the  attaining  of 
proficiency  in  scientific  domains. 

NIH-02 

Curriculum  or  Instructional  Practice 

To  support  studies  that  can  inform  the  design  of  evidence-based,  instructional  interventions. 

NIH-02 

Equity:  Learning  Difficulties 

To  develop  effective  instructional  methods  for  mitigating  learning  difficulties  in  the  attaining  of 
proficiency  in  scientific  domains. 

NIH-02 

Equity:  Learning  Difficulties 

To  find  ways  to  address  the  kinds  of  learning  difficulties  that  may  arise  in  the  attaining  of 
proficiency  in  scientific  domains. 

NIH-02 

Student  Cognition  &  Learning 

To  delineate  skill  sets  needed  to  attain  proficiency  in  scientific  domains. 

NIH-02 

Student  Cognition  &  Learning 

To  encourage  basic  and  intervention  research  in  scientific  reasoning,  learning,  and  discovery. 

NIH-02 

Student  Cognition  &  Learning 

To  encourage  research  on  causal  thinking  and  inference,  theory-evidence  coordination,  and 
reasoning  about  data. 

NIH-02 

Student  Cognition  &  Learning 

To  encourage  research  on  factors  contributing  to  conceptual  change,  as  are  studies  of  inductive  and 
deductive  reasoning,  and  the  acquisition  of  scientific  concepts  such  as  experimental  control  and 
falsibility. 

NIH-02 

Student  Cognition  &  Learning 

To  improve  our  understanding  of  the  cognitive  and  developmental  bases  of  scientific  thinking  and 
learning. 

NIH-02 

Student  Cognition  &  Learning 

To  investigate  developmental  changes  in  naive  thinking  about  the  biological  and  physical  worlds. 

NIH-02 

Student  Cognition  &  Learning 

To  investigate  individual  differences  that  may  moderate  achievement  in  science. 

NSF-01 

Student  Cognition  &  Learning 

To  support  studies  that  explore  a  variety  of  influences  on  normal  and  atypical  development  in 
science  learning  and  cognition,  including  neurobiological  substrates,  as  well  as  cognitive,  linguistic, 
sociocultural,  and  instructional  factors. 

NSF-01 

Capacity  Building 

To  build  capacity  (people  and  tools)  in  evaluation. 

NSF-01 

Evaluation  &  Assessment 

To  seek  proposals  that  offer  unique  approaches  to  evaluation  practice  in  the  generation  of  knowledge 
for  the  STEM  education  community. 

NSF-01 

Evaluation  &  Assessment 

To  build  a  strong  research  base  in  evaluation. 

NSF-02 

STEM  Education  Policies 

To  seek  proposals  that  offer  unique  approaches  for  broad  policymaking  within  the  research  and 
education  enterprise. 

NSF-02 

Curriculum  or  Instructional  Practice 

To  investigate  the  effectiveness  of  interventions  designed  to  improve  student  learning  and 
achievement  in  pre-K-12  science  and  mathematics  with  an  emphasis  on  middle  and  high  school. 

NSF-02 

Evaluation  &  Assessment 

To  develop  and  document  the  psychometric  properties  of  test  items  that  are  designed  to  measure 
learning  critical  to  scaling  up  research,  e.g.,  to  develop  measures  that  assess  the  fidelity  of 
implementations,  student  knowledge,  teacher  knowledge,  or  other  important  predictor  or  outcome 
variables  related  to  scale-up.  The  study  of  measures  that  use  technology  as  an  essential  component 
is  especially  encouraged,  e.g.,  a  variety  of  concept  inventories  are  currently  available  in  several 
science  disciplines;  studies  of  their  reliability  and  validity  are  important  before  they  can  be  used 
widely  for  scaling  up  research. 

NSF-02 

Schoolwide  &  Systemwide  Reforms 

To  identify  the  conditions  under  which  effective,  evidence-based  interventions  to  improve  preK-12 
student  learning  and  achievement  succeed  when  applied  on  a  large  scale. 

NSF-02 

STEM  Education  Policies 

To  provide  State  and  local  policy  makers,  as  well  as  school-level  administrators  and  university 
faculty  and  administrators,  with  information  on  efforts  at  improvement  that  have  led  to  increased 
and  sustained  student  learning. 

NSF-02 

Student  Cognition  &  Learning 

To  focus  on  mathematics  and  science  at  the  middle  and  high  school  levels. 

NSF-03 

Student  Cognition  &  Learning 

To  investigate  the  effectiveness  of  interventions  designed  to  improve  student  learning  and 
achievement  in  pre-K-12  science  and  mathematics  with  an  emphasis  on  middle  and  high  school. 

NSF-03 

Student  Cognition  &  Learning 

To  advance  a  rigorous  understanding  of  how  the  human  brain  supports  thought,  perception,  affect, 
action,  social  processes,  and  other  aspects  of  cognition  and  behavior.  Topics  may  bear  on  core 
functions  such  as  sensory,  learning,  language,  reasoning,  emotion,  and  executive  processes,  or  more 
specialized  processes  such  as  empathy,  creativity,  representation  of  self  and  other,  or  intentionality, 
among  many  other  possibilities.  Topics  may  also  include  how  such  processes  develop  and  change  in 
the  brain.  (Cognitive  Neuroscience  Program) 

NSF-04 

Student  Cognition  &  Learning 

To  support  studies  that  increase  our  understanding  of  cognitive,  linguistic,  social,  cultural,  and 
biological  processes  related  to  children's  and  adolescents'  development.  (Developmental  and 

Learning  Sciences  Program). 

NSF-04 

Curriculum  or  Instructional  Practice 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school 
levels  by  supporting  curriculum  development. 

NSF-04 

Faculty  Development 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school 
levels  by  internships  and  field  experiences  for  faculty,  teachers,  and  students;  and  other  activities. 

NSF-04 

Faculty  Development 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school 
levels  by  supporting  the  preparation  and  professional  development  of  college  faculty. 

NSF-04 

STEM  Workforce  Improvement 

To  strengthen  the  education  of  technicians  for  careers  in  biotechnology,  environmental  technology, 
information  technology,  manufacturing  and  many  other  science-and  engineering-related  fields  that 
drive  our  nation’s  economy. 

NSF-04 

Student  Cognition  &  Learning 

To  improve  technician  education  with  an  emphasis  on  two-year  colleges. 

NSF-04 

Teacher  Development 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school 
levels  by  internships  and  field  experiences  for  faculty,  teachers,  and  students;  and  other  activities. 

NSF-05 

Teacher  Development 

To  promote  improvement  in  technological  education  at  the  undergraduate  and  secondary  school 
levels  by  supporting  the  preparation  and  professional  development  of  secondary  school  teachers. 

NSF-05 

Curriculum  or  Instructional  Practice 

To  conduct  research  on  STEM  teaching  and  learning,  create  new  learning  materials  and  teaching 
strategies,  develop  faculty  expertise,  implement  educational  innovations,  assess  learning,  and 
evaluate  innovations. 

NSF-05 

Curriculum  or  Instructional  Practice 

To  serve  faculty,  departments,  administrators,  and  education  officials  interested  in  the  measurement 
of  student  achievement  in  courses,  curricula,  programs  of  study,  and  the  cumulative  undergraduate 
experience. 

NSF-05 

Evaluation  &  Assessment 

To  develop  new  assessment  materials  (tools)  and  processes  for  use  in  single  or  multiple 
undergraduate  disciplines. 

NSF-06 

Student  Cognition  &  Learning 

To  serve  faculty,  departments,  administrators,  and  education  officials  interested  in  the  measurement 
of  student  achievement  in  courses,  curricula,  programs  of  study,  and  the 
cumulative  undergraduate  experience. 

NSF-06 

Equity:  Gender 

To  broaden  the  participation  of  girls  and  women  in  STEM  education  by  supporting  research, 
dissemination  of  research,  and  integration  of  proven  good  practices  that  will  ultimately  lead  to  a 
larger  and  more  diverse  domestic  science  and  engineering  workforce. 

NSF-06 

Equity:  Gender 

To  build  a  strong  research  base  in  gender  differences  in  STEM. 

NSF-06 

Equity:  Gender 

To  adapt  and  replicate  proven  approaches  in  new  settings  via  supplements  to  other  NSF  grants  that 
have  not  addressed  issues  related  to  gender. 

NSF-06 

STEM  Workforce  Improvement 

To  contribute  to  the  knowledge  base  addressing  gender-related  differences  in  learning  and  in  the 
educational  experiences  that  affect  student  interest,  performance,  and  choice  of  careers. 

NSF-06 

STEM  Workforce  Improvement 

To  broaden  the  participation  of  girls  and  women  in  STEM  education  by  supporting  research, 
dissemination  of  research,  and  integration  of  proven  good  practices  that  will  ultimately  lead  to  a 
larger  and  more  diverse  domestic  science  and  engineering  workforce. 

NSF-06 

Teacher  Development 

To  contribute  to  the  knowledge  base  on  how  pedagogical  approaches  and  teaching  styles, 
curriculum,  student  services,  and  institutional  culture  contribute  to  causing  or  closing  gender  gaps 
that  persist  in  certain  fields. 

NSF-07 

Teacher  Development 

To  adapt  and  replicate  proven  approaches  in  new  settings  via  supplements  to  other  NSF  grants  that 
have  not  addressed  issues  related  to  gender. 

NSF-07 

Equity:  Disabilities 

To  increase  the  participation  and  achievement  of  persons  with  disabilities  in  science,  technology, 
engineering,  and  mathematics  (STEM)  education  and  careers. 

NSF-07 

Faculty  Development 

To  support  research  on  graduate  education 

NSF-08 

Student  Cognition  &  Learning 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  in 
science,  technology,  engineering,  and  mathematics  (STEM)  education. 

NSF-08 

Schoolwide  &  Systemwide  Reforms 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  and 
to  STEM  education  with  research  across  a  continuum  that  includes  the  diffusion  of  STEM 
innovations. 

NSF-08 

STEM  Education  Policies 

To  capitalize  on  important  developments  across  a  wide  range  of  fields  related  to  human  learning  and 
to  STEM  education  with  research  across  a  continuum  that  include  STEM  policy  research. 

NSF-08 

Student  Cognition  &  Learning 

To  support  research  across  a  continuum  that  includes  behavioral,  cognitive,  affective  and  social 
aspects  of  human  learning. 

NSF-09 

Student  Cognition  &  Learning 

To  support  research  across  a  continuum  that  includes  changing  educational  systems  to  improve 

STEM  learning. 

NSF-09 

Faculty  Development 

To  discover  successful  strategies  useful  to  educators  for  promoting  broader  adoption  or  adaptation  of 
the  factors(s)  related  to  students  receiving  associate,  baccalaureate,  of  graduate  degrees  in  STEM 
fields  of  study. 

NSF-09 

Faculty  Development 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating 
associate  and/or  baccalaureate  degree  attainment  in  STEM. 

NSF-09 

Faculty  Development 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating 
persistence  to  STEM  graduate  study. 

NSF-09 

STEM  Workforce  Improvement 

To  increase  the  number  of  students  (U.S.  citizens  or  permanent  residents)  receiving  associate  or 
baccalaureate  degrees  in  established  or  emerging  fields  within  science,  technology,  engineering,  and 
mathematics  (STEM). 

NSF-09 

STEM  Workforce  Improvement 

To  support  educational  research  projects  of  associate  or  baccalaureate  degree  attainment  in  STEM. 

NSF-10 

STEM  Workforce  Improvement 

To  generate  compelling  evidence  of  an  important  factor  or  factors  and  its  role(s)  in  facilitating 
undergraduate  access  to  STEM  careers. 

NSF-10 

Teacher  Development 

To  address  critical  issues  and  needs  regarding  the  recruitment,  preparation,  induction,  retention,  and 
life-long  development  of  K-12  science,  technology,  engineering,  and  mathematics  (STEM)  teachers. 

NSF-10 

Teacher  Development 

To  improve  the  quality  and  coherence  of  teacher  learning  experiences  across  the  continuum  through 
research  that  informs  teaching  practice  and  the  development  of  innovative  resources  for  the 
professional  development  of  K-12  STEM  teachers. 

NSF-10 

Teacher  Development 

To  promote  the  quality  of  K-12  teachers  in  a  coherent  way  throughout  teachers'  careers  with  work 
that  acknowledges  the  continuous  process  of  teachers  learning  rather  than  as  a  set  of  unrelated 
efforts. 

NSF-11 

Teacher  Development 

To  synthesize  and  further  advance  a  compelling  body  of  research  that  will  both  inform  and 
strengthen  STEM  teacher  effectiveness  and  classroom  instruction. 

NSF-11 

Capacity  Building 

To  increase  and  diversify  the  cadre  of  national  leaders  of  K-12  STEM  education  through  innovative 
programs  for  doctoral  and  postdoctoral  students. 

NSF-11 

Capacity  Building 

To  have  each  Center  organize  around  a  significant  national  question  and  provide  doctoral  and  post¬ 
doctoral  programs  around  the  priority  area. 

NSF-11 

Curriculum  or  Instructional  Practice 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning, 
teaching  strategies,  and  reform  policies  and  outcomes). 

NSF-11 

Curriculum  or  Instructional  Practices 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching, 
policies  of  educational  reform,  and  outcomes  of  standards-based  reform. 

NSF-11 

Faculty  Development 

To  increase  and  diversify  the  cadre  of  national  leaders  of  K-12  STEM  education  through  innovative 
programs  for  doctoral  and  postdoctoral  students. 

NSF-11 

Faculty  Development 

To  have  some  faculty  research  focus  on  a  significant  national  question. 

NSF-11 

Schoolwide  &  Systemwide  Reforms 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching, 
policies  of  educational  reform,  and  outcomes  of  standards-based  reform. 

NSF-11 

STEM  Education  Policies 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning, 
teaching  strategies,  and  reform  policies  and  outcomes) 

NSF-11 

STEM  Education  Policies 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching, 
policies  of  educational  reform,  and  outcomes  of  standards-based  reform. 

NSF-11 

Student  Cognition  &  Learning 

To  conduct  research  into  STEM  education  issues  of  national  import  (e.g.,  the  nature  of  learning, 
teaching  strategies,  and  reform  policies  and  outcomes). 

NSF-11 

Student  Cognition  &  Learning 

To  provide  substantive  research  opportunities  into  the  nature  of  learning,  strategies  of  teaching, 
policies  of  educational  reform,  and  outcomes  of  standards-based  reform. 

NSF-11 

Teacher  Development 

To  increase  the  number  of  K-16  educators  capable  of  delivering  high-quality  STEM  instruction  and 
assessment 

NSF-11 

Teacher  Development 

To  focus  on  the  advanced  preparation  of  science,  technology,  engineering,  and  mathematics  (STEM) 
educators,  as  well  as  the  establishment  of  meaningful  partnerships  among  education  stakeholders, 
especially  Ph.D. -granting  institutions,  school  systems,  and  informal  education  performers. 

NSF-12 

Teacher  Development 

To  renew  and  diversify  the  cadre  of  educators  in  STEM  education  in  both  formal  and  informal 
settings. 

NSF-12 

Curriculum  or  Instructional  Practice 

To  encourage  studies  of  the  effectiveness  and  impact  of  instructional  materials  developed  with  NSF 
support. 

NSF-12 

Evaluation  &  Assessment 

Research  can  range  from  meta-analyses  of  existing  students  to  large-scale  studies  involving  the 
design  and  administration  of  new  evaluation  tools. 

NSF-12 

Student  Cognition  &  Learning 

To  support  studies  that  examine  the  effectiveness  and  impact  of  NSF  funded  efforts  to  enhance 
teachers'  and  students'  STEM  learning. 

NSF-13 

Teacher  Development 

To  support  studies  that  examine  the  effectiveness  and  impact  of  NSF  funded  efforts  to  enhance 
teachers'  and  students'  STEM  learning 

NSF-13 

Curriculum  or  Instructional  Practice 

To  contribute  to  the  understanding  of  the  processes  that  support  improvement  of  K-12  mathematics 
and  science  teaching  and  learning  using  Math  and  Science  Partnership  projects  as  research  sites. 

NSF-13 

Evaluation  &  Assessment 

To  develop  new  models  and  tools  for  documenting  Math  and  Science  Partnership  projects'  progress 
toward  their  goals. 

NSF-13 

Faculty  Development 

Projects  must  address  one  of  the  MSP  Key  Features:  Teacher  quantity,  quality  and  diversity; 
challenging  courses  and  curricula;  institutional  change  and  sustainability  (...to  ensure  coordinated 
institutional  change  at  the  college  and  university  and  at  the  K-12  levels);  partnership-driven,  and 
evidence  based  design  and  outcomes." 

NSF-13 

Student  Cognition  &  Learning 

To  contribute  to  the  understanding  of  the  processes  that  support  improvement  of  K-12  mathematics 
and  science  teaching  and  learning  using  Math  and  Science  Partnership  projects  as  research  sites. 

NSF-13 

Teacher  Development 

Projects  must  address  one  of  the  MSP  Key  Features:  Teacher  quantity,  quality  and  diversity; 
challenging  courses  and  curricula;  institutional  change  and  sustainability  (...to  ensure  coordinated 
institutional  change  at  the  college  and  university  and  at  the  K-12  levels);  partnership-driven,  and 
evidence  based  design  and  outcomes." 
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NSF-14 

Capacity  Building 

To  enable  research  communities  that  can  capitalize  on  new  opportunities  and  discoveries  and 
respond  to  new  challenges. 

NSF-14 

Capacity  Building 

To  provide  innovative  educational,  research,  and  career  development  opportunities  for  all  center 
participants. 

NSF-14 

Equity:  Disabilities 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

NSF-14 

Equity:  Gender 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

NSF-14 

Equity:  Race/Ethnicity 

To  incorporate  diverse  teams  at  all  organizational  levels  of  the  center,  inclusive  of  women  and  men, 
underrepresented  minorities,  and  persons  with  disabilities. 

NSF-14 

Faculty  Development 

To  provide  innovative  educational,  research,  and  career  development  opportunities  for  all  center 
participants. 

NSF-14 

Schoolwide  &  Systemwide  Reforms 

To  understand  what  learning  is  and  how  it  is  affected  at  all  levels,  ranging  from  the  digital  to  the 
societal. 

NSF-14 

STEM  Workforce  Improvement 

To  incorporate  an  integrated,  multidisciplinary  research  program  focused  on  a  clear  intellectual  core 
and  connected  to  specific  scientific,  technological,  educational  and/or  workforce  challenges. 

NSF-14 

Student  Cognition  &  Learning 

To  advance  the  frontiers  of  all  the  sciences  of  learning  through  integrated  research. 

NSF-14 

Student  Cognition  &  Learning 

To  understand  what  learning  is  and  how  it  is  affected  at  all  levels,  ranging  from  the  digital  to  the 
societal. 

US  Department  of  Education 

ED- 1  Preschool  Curriculum  Evaluation  Research  (PCER)  Program 
ED-2  Teacher  Quality  Research  Grants  (TQR) 

ED-3  National  Center  for  Improving  Student  Learning  &  Achievement  in  Math  &  Science(closed  2/2004) 
ED-4  Interagency  Education  Research  Initiative  (IERI) 

ED-5  Cognition  &  Student  Learning  (CASL)  Research  Grants 
ED-6  Mathematics  &  Science  Education  Research  Grants 

ED-7  Consortium  for  Policy  Research  in  Education  Study  of  Instructional  Improvement 
ED-8  Field  Initiated  Evaluations  of  Education  Interventions  (formerly  Field  Initiated  Studies) 

ED-9  Development,  Implementation  &  Eval  of  Academic  Instruction  for  After-School  Prog 

ED- 10  Evaluation  of  the  Effectiveness  of  Educational  Technology  Intervention 

ED-1 1  Research  and  Innovation 

ED- 1 2  Steppingstones  of  T echnology  Innovation 

ED- 13  Research  Institutes  on  Technology 

ED- 14  Research  on  Educational  Captioning 

ED- 1 5  Pre-Doctoral  Interdisciplinary  Research  Training  Fellowship  Program 
ED- 1 6  Post-Doctoral  Research  Training  Program 


National  Institutes  of  Health 

NIH-1  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Math 

NIH-2  NICHD  Mathematics  &  Science  Cognition  &  Learning:  Development  and  Disorders:  Science 

National  Science  Foundation 

NSF-1  Evaluative  Research  &  Evaluation  Capacity  Building  (EREC) 

NSF-2  Interagency  Education  Research  Initiative 

NSF-3  Various  programs  in  Social  Behavioral  and  Economic  Sciences  (SBE) 

NSF-4  Advanced  Technological  Education  (ATE) 

NSF-5  Course,  Curriculum  &  Laboratory  Improvement/Assessment  of  Student  Achievement 
NSF-6  Research  on  Gender  in  Science  &  Engineering  (GSE) 

NSF-7  Research  Disabilities  Education  (RDE) 

NSF-8  Research  on  Learning  &  Education  (ROLE) 

NSF-9  Science,  Technology,  Engineering  and  Mathematics  Talent  Expansion  Program  (STEP) 
NSF-10  Program  Teacher  Professional  Continuum  (TPC) 

NSF-1 1  Centers  for  Learning  and  Teaching 

NSF-1 2  Instructional  Materials  Development  (IMD)- Applied  Research  Component 

NSF-1 3  Research,  Evaluation  and  Technical  Assistance  (RET A) 

NSF-1 4  Science  of  Learning  Centers  (SLC) 
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STEM  Education  Research  -  Appendix  E 

Appendix  E:  Directory  of  STEM  Education  Research,  Policy,  and  Workforce  Reports 


Report 

# 

DATE 

TITLE 

AUTHOR 

WEB  ADDRESS 

1 

1995 

Fostering  the  Use  of  Educational  Technology: 
Elements  of  a  National  Strategy. 

Thomas  Keith 

Glennan,  Arthur 
Melmed 

http://www.rand.org/publications/MR/MR682/ 

2 

1995 

Reshaping  the  Graduate  Education  of 

Scientists  and  Engineers 

National  Academy  of 
Sciences/Engineering 
Institute  of  Medicine 

http://www.nap.edu/readingroom/books/grad/ 

3 

1996 

From  Analysis  to  Action:  Undergraduate 
Education  in  Science,  Mathematics, 

Engineering,  and  Technology 

NRC 

http://books.nap.edu/openbook/NI000012/html/index.html 

4 

1997 

Computers  and  Classrooms:  The  Status  of 
Technology  inU.S.  Schools 

Richard  Coley,  et  al. 

ftp://ftp.ets.org/pub/res/compclss.pdf 

5 

1997 

Investing  in  Our  Future:  A  National  Research 
Initiative  for  America's  Children  for  the  21st 
Century, 

OSTP 

http://clinton3.nara.gov/WH/EOP/OSTP/Children/ 

6 

1997 

Report  to  the  President  on  the  Use  of 
Technology  to  Strengthen  K-12  Education  in 
the  United  States 

OSTP 

http://www.ostp.gov/PCAST/k-12ed.html 

7 

1997 

Science  Teaching  Reconsidered:  A  Handbook 

Committee  on 
Undergraduate 

Science  Education, 
National  Research 
Council 

http://www.nap.edu/catalog/5287.html 

8 

1997 

Toward  Inequality:  Disturbing  Trends  in 

Higher  Education 

Paul  E.  Barton 

http://www.ets.org/research/pic/twtoc.html 

9 

1998 

Breaking  the  Social  Contract:  The  Fiscal 

Crisis  in  California  Higher  Education 

Roger  Benjamin  and 
Stephen  J.  Carroll 
(RAND) 

http://www.rand.org/publications/IP/IP  1 73/index.html 

10 

1998 

Elements  of  a  National  Strategy  to  Foster 
Effective  Use  of  Technology  in  Elementary 
and  Secondary  Education 

Thomas  K.  Glennan, 
Jr., 

http://www.rand.org/publications/CT/CT145/CT145.pdf 

11 

1998 

Failing  Our  Children:  Implications  of  the 

Third  International,  Mathematics  and  Science 
Study 

NSF 

http://www.nsf.gov/nsb/documents/1998/nsb98 1 54/nsb98 1 54.htm 

12 

1998 

National  Directions  in  Education  Research 
Planning.  A  Conference  Co-Sponsored  by  the 
National  Educational  Research  Policy  and 

Michael  Timpane, 

http://www.ed.gov/pubs/nationaldirections/ 
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Priorities  Board  and  the  Office  of  Educational 
Research  and  Improvement,  U.S.  Department 
of  Education 

13 

1999 

Closing  the  Education  Gap: 

Benefits  and  Costs 

Georges  Vemez, 
Richard  A.  Krop,  C. 
Peter  Rydell 

httD://www.raiid.ore/publications/MR/MR1036/index.html 

14 

1999 

High  Stakes:  Testing  for  Tracking,  Promotion, 
and  Graduation 

Jay  P.  Heubert  and 
Robert  M.  Hauser, 
Editors;  Committee 
on  Appropriate  Test 
Use,  National 

Research  Council 

httD://www.iiap.edu/books/0309062802/html/index.html 

15 

1999 

Making  Money  Matter:  Financing  America's 
Schools 

Commission  on 
Behavioral  and 

Social  Sciences  and 

Education 

(CBASSE) 

httn://books. naD.edu/onenbook/0309065283/html/index.html 

16 

1999 

Preparing  our  Children:  Math  and  Science 
Education  in  the  National  Interest 

NSF 

http://www.nsf.aov/pubs/1999/nsb9931/start.htm 

17 

2000 

A  Study  of  K-12  Mathematics  and  Science 
Education  in  the  United  States 

Horizon  Research 

Inc 

httD://www.horizon-research.com/insidetheclassroom/reDorts/lookin2/comDlete.Ddf 

18 

2000 

Bridging  Disciplines  in  the  Brain,  Behavioral, 
and  Clinical  Sciences 

Institute  of  Medicine 
(IOM) 

httD://www.naD.edu/books/0309070783/html/index.html 

19 

2000 

Educating  Teachers  of  Science,  Mathematics, 
and  Technology:  New  Practices  for  the  New 
Millennium 

Committee  on 

Science  and 
Mathematics  Teacher 
Preparation,  National 
Research  Council 

httD://www.naD.edu/books/0309070333/html 

20 

2000 

Enhancing  the  Postdoctoral  Experience  for 
Scientists  and  Engineers:  A  Guide  for 
Postdoctoral  Scholars,  Advisers,  Institutions, 
Funding  Organizations,  and  Disciplinary 
Societies 

National  Academy  of 
Sciences,  National 
Academy  of 
Engineering,  Institute 
of  Medicine 

httD://www.naD.edu/books/0309069963/html/ 

21 

2000 

Graduate  Education  in  the  Chemical  Sciences: 
Issues  for  the  21st  Century:  Report  of  a 
Workshop 

Chemical  Sciences 
Roundtable,  Board  of 
Chemical  Sciences 
and  Technology, 
National  Research 
Council 

httD://www.naD.edu/books/0309071305/html/index.html 

22 

2000 

How  People  Learn:  Brain,  Mind,  Experience, 
and  School:  Expanded  Edition 

Committee  on 
Developments  in  the 
Science  of  Learning 
with  additional 
material  from  the 

http://www.nap.edu/books/0309070368/html/ 
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Committee  on 

Learning  Research 
and  Educational 
Practice,  National 
Research  Council 

23 

2000 

How  Teaching  Matters:  Bringing  the 

Classroom  Back  Into  Discussions  of  Teacher 
Quality 

Harold  Wenglinsky 

http://www.ets.org/research/pic/teamat.pdf 

24 

2000 

Teaching  Practices  and  Student  Achievement: 
Report  of  First-Year  Findings  from  the 
'Mosaic'  Study  of  Systemic  Initiatives  in 
Mathematics  and  Science 

Stephen  P.  Klein, 

Laura  Hamilton, 

Daniel  McCaffrey, 
Brian  Steelier,  Abby 
Robyn,  Delia 
Burroughs 

http://www.rand.org/publications/MR/MRl  233/index,  html 

25 

2000 

The  American  Community  College  Turns  100: 

A  Look  at  its  Students,  Programs,  and 

Prospects 

Richard  J.  Coley, 

ETS 

httn://www.ets.ora/research/nic/cc.Ddf 

26 

2000 

What  Jobs  Require:  Literacy,  Education,  and 
Training,  1940-2006 

Paul  E.  Barton,  ETS 

http://www.ets.org/research/pic/iobs.ndf 

27 

2001 

Adding  It  Up:  Helping  Children  Learn 
Mathematics 

Mathematics 

Learning  Study 
Committee,  National 
Research  Council 

http://www.nan.edu/books/0309069955/html/index.html 

28 

2001 

Differences  in  the  Gender  Gap:  Comparisons 
Across  Racial/Ethnic  Groups  in  Education  and 
Work 

Richard  J.  Coley, 

ETS 

http://www.ets.org/research/pic/gender.pdf 

29 

2001 

Facing  the  Hard  Facts  in  Education  Reform 

Paul  E.  Barton,  ETS 

http://www.ets.org/research/nic/facingfacts.pdf 

30 

2001 

From  Scarcity  to  Visibility:  Gender 

Differences  in  the  Careers  of  Doctoral 

Scientists  and  Engineers 

J.  Scott  Long, 

Editors;  Panel  for  the 
Study  of  Gender 
Differences  in  Career 
Outcomes  of  Science 
and  Engineering 
Ph.D.s,  Committee 
on  Women  in 

Science  and 
Engineering, 

National  Research 
Council 

http :  //www  .nap .  edu/books/03 090 5  5  80 6/html/ 

31 

2001 

Knowing  and  Learning  Mathematics  for 
Teaching:  Proceedings  of  a  Workshop 

Center  for  Education 

http://www.nap.edu/books/0309072522/html/ 

32 

2001 

Knowing  What  Students  Know:  The  Science 
and  Design  of  Educational  Assessment 

Committee  on  the 
Foundations  of 
Assessment,  James 

http://www.nap.edu/catalog/10019.html 

84 


W.  Pellegrino, 

Naomi  Chudowsky, 
and  Robert  Glaser, 
editors,  Board  on 
Testing  and 
Assessment,  Center 
for  Education, 

National  Research 
Council 

33 

2001 

Private  Giving  to  Public  Schools  and  Districts 
in  Los  Angeles  County: 

A  Pilot  Study 

Ron  Zimmer,  Cathy 
Krop,  Tessa 

Kaganoff,  Karen  E. 
Ross,  Dominic 

Brewer 

httn://www.rand.ora/Dublications/MR/MRl  429/index,  html 

34 

2001 

Testing  Teacher  Candidates:  The  Role  of 
Licensure  Tests  in  Improving  Teacher  Quality 

CFE  (Center  for 
Education) 

httn://books. naD.edu/onenbook/0309074207/html/index.html 

35 

2001 

Trends  in  Federal  Support  of  Research  and 
Graduate  Education 

Board  on  Science, 
Technology,  and 
Economic  Policy 
(STEP) 

httD://books.  nan.edu/onenbook/0309075890/html/index.html 

36 

2002 

A  Life  of  the  Mind  for  Practice:  Professional 
Education  and  the  Liberal  Arts 

William  Sullivan 

httn://www.cameaiefoundation.ora/elibrarv/docs/Cross-Prof  Seminar.ndf 

37 

2002 

Enhancing  Undergraduate  Learning  with 
Information  Technology:  A  Workshop 

Summary 

NRC,  Center  for 
Education 

httn://www.nan.edu/books/0309082781/html/ 

38 

2002 

Learning  and  Understanding:  Improving 
Advanced  Study  of  Mathematics  and  Science 
in  U.S.  High  Schools 

Center  for  Education 

httn://www.naD.edu/onenbook/0309074401/html/ 

39 

2002 

Meeting  the  Need  for  Scientists,  Engineers, 
and  an  Educated  Citizenry  in  a  Technological 
Society 

Paul  E.  Barton,  ETS 

httn://www.ets.or2/research/nic/meetinaneed.ndf 

40 

2002 

The  Making  of  Differences:  A  Table  of 
Learning 

Lee  Shulman 

httn://www.cameaiefoundation.ora/elibrarv/docs/makina  differences.htm 

41 

2002 

Merging  University  Students  into  K-12 

Science  Education  Reform 

Valerie  L.  Williams 
(RAND) 

httD://www.rand.ora/nublications/MR/MR1446/MR1446.Ddf 

42 

2002 

Preparing  Our  Teachers:  Opportunities  for 
Better  Reading  Instruction 

Dorothy  Strickland, 
Catherine  Snow,  Peg 
Griffin,  M.  Susan 
Bums,  Peggy 
McNamara 

httn://www.nan.edu/books/0309074452/html/ 

43 

2002 

Raising  Achievement  and  Reducing  Gaps: 
Reporting  Progress  Toward  Goals  for 

Academic  Achievement  in  Mathematics 

Paul  E.  Barton,  ETS 

httn://www.ets.org/research/nic/raising.ndf 
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44 

2002 

Scientific  Research  in  Education 

CFE 

httD://books.naD.edu/oDenbook/0309082919/html/index.html 

45 

2002 

Situating  the  Scholarship  of  Teaching  and 
Learning:  A  Cross-Disciplinary  Conversation 

Mary  Taylor  Huber 
and  Sherwyn 

Morreale 

http://www.camegiefoundation.org/elibrarv/docs/situating.htm 

46 

2002 

The  Knowledge  Economy  and  Postsecondary 
Education:  Report  of  a  Workshop 

CFE 

httn://books.  naD.edu/onenbook/0309082927/html/index.html 

47 

2003 

Closing  the  College  Participation  Gap:  A 
National  Summary 

ECS 

http://www.ecs.org/clearinghouse/47/84/4784.pdf 

48 

2003 

Information  Technology  (IT)-Based 

Educational  Materials:  Workshop  Report  with 
Recommendations 

NRC 

httn://www.nat).edu/books/0309089743/html/ 

49 

2003 

Eight  Questions  on  Teacher  Preparation:  What 
does  the  Research  Say? 

Michael  B.  Allen, 
Education 

Commission  of  the 
States 

http://www.ecs.org/html/educationIssues/teachingciualitv/tpreport/home/summarv.pdf 

50 

2003 

Hispanics  in  Science  and  Engineering:  A 

Matter  of  Assistance  and  Persistence, 

Paul  E.  Barton,  ETS 

http://www.ets.org/research/pic/hispanic.pdf 

51 

2003 

Learning  for  the  21st  Century 

Partnership  for  21st 
Century  Skills 

httn://2  lstcenturvskills.org/downloads/P21  Renort.ndf 

52 

2003 

Mathematical  Proficiency  for  All  Students: 
Toward  a  Strategic  Research  and 

Development  Program  in  Mathematics 
Education 

RAND  Mathematics 
Study  Panel, 

Deborah 

Loewenberg  Ball, 
Chair 

httD://www.rand.ora/Dublications/MR/MR1643/ 

53 

2003 

Science  and  Engineering  Infrastructure  for  the 
21st  Century:  The  Role  of  the  National 

Science  Foundation 

NSF 

httn://www.nsf.  aov/nsb/documents/2002/nsb02 1 90/nsb02 1 90.ndf 

54 

2003 

Strategic  Education  Research  Partnership 

Education  Research 
Partnership,  National 
Research  Council 

httn://www.nat).edu/books/0309088798/html/ 

55 

2003 

What  Is  the  Influence  of  the  National  Science 
Education  Standards?:  Reviewing  the 

Evidence,  A  Workshop  Summary 

NRC:  CFE  (Center 
for  Education) 

http://www.nap.edu/openbook/0309087430/html/ 

56 

2004 

A  Bridge  for  All: 

Higher  Education  Design  Principles  to 

Broaden  Participation  in  Science,  Technology, 
Engineering  and  Mathematics 

BEST  (Building 
Engineering  & 

Science  Talent) 

http://bestworkforce.org/PDFdocs/BEST  BridgeforAll  HighEdFINAL.pdf 

57 

2004 

Integrative  Learning:  Mapping  the  Terrain 

Mary  Taylor  Huber, 
Pat  Hutching 

http://www.camegiefoundation.org/elibrarv/docs/Mapping  Terrain.pdf 

58 

2004 

On  Evaluating  Circular  Effectiveness:  Judging 
the  Quality  of  K-12  Mathematics  Evaluations 

Mathematical 

Sciences  Education 
Board  (MSEB), 

Center  for  Education 
(CFE) 

http://nap.edu/books/0309092426/html 
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59 

2004 

Quiet  Crisis:  Falling  Short  in  Producing 
American  Scientific  and  Technical  Talent 

BEST  (Building 
Engineering  & 

Science  Talent) 

httn://bestworkforce.ora/PDFdocs/Ouiet  Crisis. ndf 

60 

2004 

Reforming  Teacher  Education:  A  First  Year 
Progress  Report  on  Teachers  for  a  New  Era 

Sheila  Nataraj  Kirby, 
Jennifer  Sloan 
McCombs,  et  al. 

httn://www.rand.or2/Dublications/TR/TR149/TR149.Ddf 

61 

2004 

The  Talent  Imperative:  Meeting  American’s 
challenge  in  science  and  engineering,  ASAP 

BEST  (Building 
Engineering  & 

Science  Talent 

httnV/bestworkforce.ora/PDFdocs/BESTTalentlmnerativeFINAL.Ddf 

62 

2004 

What  it  Takes: 

Pre-K-12  Design  Principles  to  Broaden 
Participation  in  Science,  Technology, 
Engineering  and  Mathematics 

BEST  (Building 
Engineering  & 

Science  Talent) 

httn://bestworkforce.ore/PDFdocs/BESTPre-K-12Rer>  nartl  Anr2004.ndf 

63 

2004 

Advancing  Scientific  Research  in  Education 

Center  for  Education 
(CFE) 

httn://www.naD.edu/books/030909321X/htmE 

64 

2005 

Mathematical  and  Scientific  Development  in 
Early  Childhood:  A  Workshop  Summary 

Mathematical 

Sciences  Education 
Board  (MSEB), 

Center  for  Education 
(CFE) 

httn://www.naD.edu/books/0309095034/htmE 
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Appendix  F:  Research  Recommendations  Selected  From  STEM  Reports 


Report  # 

STEM 

Program 

Research  Recommendation 

Relevant  Federal 

Area 

Priority/ 

Objective 

Category 

Program 

1 

Technology 

Sustaining  a  vigorous  and  relevant  program  of  research  and  development  related  to 
educational  technology. 

2 

Engineering 

STEM 

The  National  Science  Foundation  should  support  extramural  research  on  actual  career 

NSF-04 

Education 

Policy 

patterns  in  science  and  engineering. 

NSF-09 

5 

Miscellaneous 

Undergirding  any  successful  research  planning  program  must  be  two  essential 
supporting  activities: 

*  perennial  attention  to  the  capacity  of  the  education  research  system:  its  human 

ED- 15 

resources,  and  its  institutions  and  financial  resources  and  inventory  and  descriptive 

ED- 16 

analysis  of  the  system  would  be  a  good  first  step;  and 

*  an  entirely  new  approach  to  communications,  taking  the  word  on  educational 
research  to  the  profession,  the  public,  and  the  policy  world. 

NSF-8 

6 

Technology 

Schoolwide  & 

Particular  attention  should  be  given  to  exploring  the  potential  role  of  technology  in 

ED- 10 

Systemwide 

achieving  the  goals  of  current  educational  reform  efforts  through  the  use  of  new 

NSF-2 

Reforms 

pedagogic  methods  based  on  a  more  active,  student-centered  approach  to  learning  that 
emphasizes  the  development  of  higher-order  reasoning  and  problem-solving  skills. 

NSF-4 

11 

Engineering 

Faculty 

To  develop  a  much-needed  consensus  on  a  common  core  of  mathematics  and  science 

Maybe  NSF-11 

Development 

knowledge  and  skills  to  be  embedded  consistently  in  instructional  materials; 

NIH-01 

NIH-02 

12 

Math 

Candidates  for  the  short  list  of  research  priorities  seemed  rather  obvious:  continued 
focus  on  reading  and  language  learning;  expanded  attention  to  mathematics;  the 
dynamics  of  teacher  performance  and  effectiveness  in  schools  and  classrooms;  and  new 
emphasis  on  technology  and  telecommunications,  international  studies,  and  learning  in 
family,  community,  and  workplace  settings. 

All  programs 

14 

Design/ 

Research  is  needed  on  the  effects  of  high-stakes  graduation  tests  on  teaching,  learning, 

88 


Methods/ 

Measurement 

and  high  school  completion.  Research  is  also  needed  on  alternatives  to  test-based  denial 
of  the  high  school  diploma,  such  as  endorsed  diplomas,  end-of-course  tests,  and 
combining  graduation  test  scores  with  other  indicators  of  knowledge  and  skill  in 
making  the  graduation  decision. 

14 

Design/ 

Methods/ 

Measurement 

Equity-Learning 

Difficulties 

More  research  is  needed  to  enable  students  with  disabilities  to  participate  in  large-scale 
assessments  in  ways  that  provide  valid  information.  This  goal  significantly  challenges 
current  knowledge  and  technology  about  measurement  and  test  design  and  the 
infrastructure  needed  to  achieve  broad-based  participation. 

ED-12,13,14 

NSF-07 

14 

Evaluation 

Evaluation  & 
Assessment 

High-stakes  testing  programs  should  routinely  include  a  well-designed  evaluation 
component.  Policy  makers  should  monitor  both  the  intended  and  unintended 
consequences  of  high-stakes  assessments  on  all  students  and  on  significant  subgroups 
of  students,  including  minorities,  English-language  learners,  and  students  with 
disabilities. 

NSF-12 

14 

Science 

Evaluation  & 
Assessment 

Longitudinal  investigations  on  instructional  reform  and  testing. 

15 

Science 

Math 

Equity- 

Race/Ethnicity 

Identifying  the  Sources  of  Low  Achievement.  The  joint  study  by  the  National  Center 
for  Education  Statistics  and  the  National  Science  Foundation,  Understanding  Racial- 
Ethnic  Differences  in  Secondary  School  Science  and  Mathematics  Achievement, 
provides  a  useful  starting  point  for  understanding  and  addressing  the  sources  of  low 
performance. 

ED-01 

ED-02 

ED-05 

ED-06 

NIH-01 

NIH-02 

19 

Engineering 

Faculty 

Development 

Universities  whose  primary  mission  includes  education  research  should  set  as  a  priority 
the  development  and  execution  of  peer-reviewed  research  studies  that  focus  on  ways  to 
improve  teacher  education,  the  art  of  teaching,  and  learning  for  people  of  all  ages.  New 
research  that  focuses  broadly  on  synthesizing  data  across  studies  and  linking  it  to 
school  practice  in  a  wide  variety  of  school  settings  would  be  especially  helpful  to  the 
improvement  of  teacher  education  and  professional  development  for  both  prospective 
and  experienced  teachers.  The  results  of  this  research  should  be  collated  and 
disseminated  through  a  national  electronic  database  or  library. 

ED-02 

NSF-13 

Conveying  research 
is  a  process 
recommendation 

22 

Math 

Curriculum  or 

Instructional 

Practice 

As  an  extension  of  project  area  1  above,  or  in  some  cases  as  a  substitute,  the 
development  and  evaluation  of  new  curriculum  and  assessment  materials  that  reflect  the 
principles  of  learning  outlined  herein  should  be  undertaken.  Again,  the  development 
should  be  done  by  teams  of  disciplinary  experts,  cognitive  scientists,  curriculum 
developers,  and  expert  teachers.  Ideally,  research  in  this  category  will  begin  with 
existing  curricula  and  modify  them  to  better  reflect  key  principles  of  learning.  In  some 
cases,  however,  exemplary  curricula  for  particular  kinds  of  subject  matter  may  not 
exist,  so  the  teams  will  need  to  create  them.  This  research  and  development  might  be 

NSF-11 

NSF-12 

NSF-14 
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coordinated  with  the  ongoing  efforts  of  the  National  Science  Foundation  to  ensure 
complementary  rather  than  duplicative  efforts. 

22 

Math 

Evaluation  & 
Assessment 

Conduct  research  on  formative  assessment.  A  separate  research  effort  on  formative 
assessment  is  recommended.  The  importance  of  making  students’  thinking  visible  by 
providing  frequent  opportunities  for  assessment,  feedback,  and  revision,  as  well  as 
teaching  students  to  engage  in  self-assessment,  is  emphasized  throughout  this  volume 
and  in  the  proposals  above.  But  the  knowledge  base  on  how  to  do  this  effectively  is  still 
weak.  To  bolster  the  understanding  of  formative  assessment  so  that  it  can  more 
effectively  be  built  into  curricula,  this  research  effort  should: 

ED-5 

NSF-1 

NSF-12 

•  Formulate  design  principles  for  formative  assessments  that  promote  the 
development  of  coherent,  well-organized  knowledge.  The  goal  of  these 
assessments  is  to  tap  understanding  rather  than  memory  for  procedures  and 
facts. 

•  Experiment  with  approaches  to  developing  in  students  and  teachers  a  view  of 
formative  assessment  and  self-assessment  as  an  opportunity  for  providing 
useful  information  that  allows  for  growth,  rather  than  as  an  outcome  measure 
of  success  or  failure. 

•  Explore  the  potential  of  new  technologies  that  provide  the  opportunity  to 
incorporate  formative  assessment  into  teaching  in  an  efficient  and  user- 
friendly  fashion. 

22 

Science 

Schoolwide  & 

Systemwide 

Reforms 

Research  done  on  effective  methods  of  communicating  reform  ideas  to  teachers, 
administrators,  and  policy  makers. 

30 

Engineering 

Equity-Gender 

A  set  of  key  measures  and  benchmarks  should  be  established  for  assessing  the  progress 
of  women  in  science  and  engineering.  An  assessment  of  progress  relative  to  these 
benchmarks  should  be  made  available  shortly  after  each  public  release  of  the  Survey  of 
Earned  Doctorates  and  the  Survey  of  Doctoral  Recipients. 

NSF-06 

30 

Engineering 

Equity-Gender 

Entry  into  the  Ph.D.:  What  accounts  for  the  lower  entry  of  women  into  some  fields? 
Given  the  progress  made  by  those  women  already  in  science,  a  clear  objective  needs  to 
be  increasing  the  number  of  women  entering  science  and  engineering.  To  fully 
understand  the  entry  of  women  into  the  Ph.D.,  studies  are  needed  of  admissions 
practices,  especially  among  top  institutions.  The  lower  representation  of  women  as 
undergraduates  in  Research  I  institutions  also  needs  to  be  more  fully  understood. 

NSF-06 

For  those  in  graduate  programs,  further  information  is  needed  on  graduate  support  and 
how  career  interruptions  for  women  affect  their  options  for  support. 

NSF-06 
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The  transition  from  the  Ph.D.  to  the  full-time  labor  force  is  a  critical  point  at  which 
relatively,  more  women  than  men  are  lost.  To  understand  this  substantial  loss  of  women 
who  have  completed  their  graduate  education,  requires  an  examination  of  postdoctoral 
fellowships  and  the  effects  of  marriage  and  family.  Our  evidence  clearly  indicates  that 
having  young  children  is  related  to  the  entry  of  women  into  the  full  time  labor  force. 

NSF-06 

Throughout  the  career,  proportionally  more  women  than  men  leave  science  and 
engineering  entirely.  More  information  is  need  on  why  these  highly  trained  scientists 
are  lost.  Here  also  constraints  imposed  by  familial  obligations,  career  interruptions,  and 
constraints  on  mobility  need  to  be  considered.  To  this  end,  the  SDR  should  be  revised 
to  collect  additional  information  particularly  relevant  to  understanding  the  loss  of  a 
disproportionate  number  of  women  from  the  full  time  S&E  labor  force.  Questions  on 
reasons  for  part  time  employment  should  be  expanded  and  new  questions  on  reasons  for 
not  being  in  the  labor  force  or  working  outside  of  S&E  should  be  added. 

NSF-09  maybe 
NSF-06 

Finally,  while  women  remain  underrepresented,  most  minority  groups  are  even  less 
well  represented.  Detailed  studies  of  the  situation  facing  minorities  are  needed.  Given 
the  small  numbers  of  minority  scientists  and  engineers,  these  studies  may  require  the 
collection  of  new  data. 

ED-04 

Maybe  NSF-04 

32 

Design/ 

Methods/ 

Measurement 

Evaluation  & 
Assessment 

Funding  should  be  provided  for  a  major  program  of  research,  guided  by  a  synthesis  of 
cognitive  and  measurement  principles,  focused  on  the  design  of  assessments  that  yield 
more  valid  and  fair  inferences  about  student  achievement!  This  research  should  be 
conducted  collaboratively  by  multidisciplinary  teams  comprising  both  researchers  and 
practitioners.  A  priority  should  be  the  development  of  models  of  cognition  and  learning 
that  can  serve  as  the  basis  for  assessment  design  for  all  areas  of  the  school  curriculum. 
Research  on  how  students  learn  subject  matter  should  be  conducted  in  actual 
educational  settings  and  with  groups  of  learners  representative  of  the  diversity  of  the 
student  population  to  be  assessed.  Research  on  new  statistical  measurement  models  and 
their  applicability  should  be  tied  to  modern  theories  of  cognition  and  learning.  Work 
should  be  undertaken  to  better  understand  the  fit  between  various  types  of  cognitive 
theories  and  measurement  models  to  determine  which  combinations  work  best  together. 
Research  on  assessment  design  should  include  exploration  of  systematic  and  fair 
methods  for  taking  into  account  aspects  of  examinees’  instructional  background  when 
interpreting  their  responses  to  assessment  tasks.  This  research  should  encompass 
careful  examination  of  the  possible  consequences  of  such  adaptations  in  high-stakes 

ED-05 

NIH-02 

(includes  some 
process 

recommendations) 

NSF-02 

NSF-08 

NSF-11 

ED- 11 
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assessment  contexts. 

32 

Design/ 

Methods/ 

Measurement 

Evaluation  & 
Assessment 

Funding  should  be  provided  for  in-depth  analyses  of  the  critical  elements  (cognition, 
observation,  and  interpretation)  underlying  the  design  of  existing  assessments  that  have 
attempted  to  integrate  cognitive  and  measurement  principles  (including  the  multiple 
examples  presented  in  this  report).  This  work  should  also  focus  on  better  understanding 
the  impact  of  such  exemplars  on  student  learning,  teaching  practice,  and  educational 
decision  making. 

NSF-02 

32 

Design/ 

Methods/ 

Measurement 

Evaluation  & 
Assessment 

Research  should  be  conducted  to  explore  how  new  forms  of  assessment  can  be  made 
practical  for  use  in  classroom  and  large-scale  contexts  and  how  various  new  forms  of 
assessment  affect  student  learning,  teacher  practice,  and  educational  decision  making. 
This  research  should  also  explore  how  teachers  can  be  assisted  in  integrating  new  forms 
of  assessment  into  their  instructional  practices.  It  is  particularly  important  that  such 
work  be  done  in  close  collaboration  with  practicing  teachers  who  have  varying 
backgrounds  and  levels  of  teaching  experience.  The  research  should  encompass  ways  in 
which  school  structures  (e.g.,  length  of  time  of  classes,  class  size,  and  opportunity  for 
teachers  to  work  together)  affect  the  feasibility  of  implementing  new  types  of 
assessments  and  their  effectiveness. 

NSF-02 

NSF-12 

NSF-13 

ED-05  (Maybe) 

32 

Design/ 

Methods/ 

Measurement 

Evaluation  & 
Assessment 

Developers  of  assessment  instalments  for  classroom  or  large-scale  use  should  pay 
explicit  attention  to  all  three  elements  of  the  assessment  triangle  (cognition, 
observation,  and  interpretation)  and  their  coordination.  All  three  elements  should  be 
based  on  modern  knowledge  of  how  students  learn  and  how  such  learning  is  best 
measured.  Considerable  time  and  effort  should  be  devoted  to  a  theory-driven  design 
and  validation  process  before  assessments  are  put  into  operational  use. 

NSF-01 

NSF-12 

NSF-05 

32 

Science 

Better  understand  the  fit  between  various  types  of  cognitive  theories  and  measurement 
models  to  determine  which  combinations  work  best  together. 

32 

Science 

Evaluation  & 
Assessment 

How  various  new  forms  of  assessment  affect  student  learning,  teacher  practice,  and 
educational  decision-making. 

32 

Science 

Evaluation  & 
Assessment 

In-Depth  analyses  of  the  critical  elements  (cognition,  observation,  and  interpretation) 
underlying  the  design  of  existing  assessments  that  have  attempted  to  integrate  cognitive 
and  measurement  principles 

32 

Science 

Evaluation  & 
Assessment 

Student  achievement  with  a  model  of  cognition  and  learning  that  can  serve  as  the  basis 
for  assessment  design  for  all  areas  of  the  school  curriculum. 

34 

Design/ 

Methods/ 

Measurement 

Systematic  research  that  investigates  the  impact  of  specific  accommodations  on  the  test 
performance  of  both  English-language  learners  and  other  students  is  needed. 
Accommodations  should  be  investigated  to  see  whether  they  reduce  construct-irrelevant 
sources  of  variance  for  English-language  learners  without  disadvantaging  other  students 
who  do  not  receive  accommodations.  The  relationship  of  test  accommodations  to 
instructional  accommodations  should  also  be  studied. 

34 

Design/ 

Methods/ 

States  should  arrange  for  independent  evaluations  of  their  current  tests  and  teacher 
licensure  systems  and  make  the  results  of  these  independent  examinations  of  their 

ED-07  maybe 
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Measurement 

systems  available  for  outside  review. 

34 

Evaluation 

It  is  important  to  collect  validity  data  that  go  beyond  content-related  validity  evidence 
for  initial  licensing  tests.  However,  conducting  high-quality  research  of  this  kind  is 
complex  and  costly.  Examples  of  relevant  research  include  investigations  of  the 
relationships  between  test  results  and  other  measures  of  candidate  knowledge  and  skills 
or  on  the  extent  to  which  tests  distinguish  candidates  who  are  at  least  minimally 
competent  from  those  who  are  not. 

NSF-10 

NSF-12 

34 

Evaluation 

Little  research  has  been  conducted  on  the  extent  to  which  scores  on  current  teacher 
licensure  tests  relate  to  other  measures  of  beginning  teacher  competence.  Much  of  the 
research  that  has  been  conducted  suffers  from  methodological  problems  that  interfere 
with  making  strong  conclusions  about  the  results.  This  makes  it  hard  to  determine  what 
effect  licensure  tests  might  have  on  improving  the  actual  competence  of  beginning 
teachers. 

NSF-10 

NSF-12 

34 

Evaluation 

Teacher 

Development 

The  committee  encourages  the  federal  government  and  others  to  conduct  research  on 
the  extent  to  which  teacher  licensure  tests  distinguish  between  beginning  teachers  who 
are  at  least  minimally  competent  and  those  who  are  not  regarding  the  knowledge  and 
skills  the  tests  are  intended  to  measure.  This  research  should  include  evidence  on  a 
broad  range  of  teacher  competencies.  Such  research  is  likely  to  improve  the 
development  of  teacher  licensure  tests.  Within  the  limits  of  privacy  law,  states  should 
make  their  raw  data  available  to  the  research  community  to  facilitate  development  and 
validity  research  on  initial  teacher  licensure  tests. 

ED-2 

NSF-10 

NSF-12 

NSF-13 

41 

Evaluation 

Evaluation  & 
Assessment 

Further  research  into  program  design,  evaluation,  and  implementation  issues  targeting 
the  needs  of  K-12  schools 

50 

What  kind  of  students  graduating  from  high  school  actually  persist  in  college? 

52 

Math 

The  research  and  education  communities  also  need  to  learn  how  children,  who  bring 
different  personal  experiences  to  school  with  them,  learn  the  mathematical  practices 
that  are  essential  to  effective  day-to-day  use  of  mathematics. 

NIH-1 

NIH-2 

52 

Math 

Capacity 

Building 

To  provide  the  necessary  knowledge  and  the  capacity  to  use  that  knowledge  in  practice, 
this  report  recommends  a  significant  program  of  research  and  development  aimed  at 
building  resources  for  improved  teaching  and  learning.  Because  resources  are  limited, 
the  panel  deliberated  at  length  to  identify  the  research  areas  that  are  most  likely  to  yield 
improved  knowledge  and  practice  and  to  attain  the  dual  goals  of  mathematical 
proficiency  and  equity  in  the  acquisition  of  proficiency. 

NSF-11 

52 

Math 

Teacher 

Development 

The  research  and  education  communities  need  to  identify  the  knowledge  that  can 
enable  teachers  to  help  their  students  develop  mathematical  proficiency,  and  they  need 
to  develop  robust  ways  of  helping  teachers  acquire  and  use  that  knowledge 

NSF  11 

NSF-12 

NSF-13 

ED-2 

ED-4 

52 

Math 

Teacher 

This  report  recommends  three  priority  focus  areas  for  programmatic  research  and 

NSF-2 
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Development 

development — developing  teachers’  mathematical  knowledge  in  ways  that  are  directly 
useful  for  teaching,  teaching  and  learning  skills  for  mathematical  thinking  and  problem 
solving,  and  teaching  and  learning  of  algebra  from  kindergarten  through  12th  grade. 

NSF  10 

NSF-11 

NSF-12 

ED-2 

ED-3 

ED-4 

54 

Technology 

Teacher 

Development 

Study  the  best  ways  to  educate,  train  and  evaluate  teachers  in  21st  century  skills 

55 

Science 

Large  scale  studies  to  investigate  the  impact  of  standards-based  science  programs  on 
student  achievement. 

55 

Science 

Schoolwide  & 

Systemwide 

Reforms 

Studies  to  determine  the  degree  to  which  local  school  districts  are  adopting  high- 
quality,  standard-based  materials 

58 

Evaluation 

Evaluation  & 
Assessment 

A  curricular  program's  effectiveness  should  be  ascertained  through  the  use  of  multiple 
methods  of  evaluation,  each  of  which  should  be  a  scientifically  valid  study.  Periodic 
synthesis  of  the  results  across  evaluations  studies  should  also  be  conducted. 

NSF-1 

61 

Science 

Equity-Gender 

More  research  on  what  works  at  all  levels  in  developing  the  talent  of  underrepresented 
groups. 

Many  programs. 

See  grid. 

62 

Science 

Equity- 

Race/Ethnicity 

Deepen  the  knowledge  base  regarding  what  works  to  prepare  and  interest  students  who 
are  underrepresented  in  math  and  science 

NSF-6 

NSF-8 

NSF-9 

NSF-11 

62 

Science 

Math 

STEM 

Education 

Policies 

The  program  evaluation  conducted  by  BEST  and  AIR  yielded  three  policy  imperatives: 
first,  to  deepen  the  knowledge  base  regarding  what  works  to  prepare  and  interest 
students  who  are  underrepresented  in  math  and  science;  second,  to  tighten  the  link 
between  knowledge,  policy  and  practice  in  this  area;  third,  to  align  targeted  programs 
and  system-wide  approaches  more  closely.  The  remainder  of  this  chapter  outlines  these 
imperatives  and  proposes  a  course  of  action  to  address  each. 

NSF-02 

NSF-13 
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STEM  Education  Research  -  Appendix  G 
Appendix  G:  Process  Recommendations  Selected  From  STEM  Reports 


Report  # 

STEM 

Area 

Program 

Priority/ 

Objective 

Category 

Process  Recommendation 

Status  with  regard 
to  Federal 

Programs 

1 

Technology 

STEM 

Education 

Policy 

The  federal  government's  role  should  involve  leadership,  funding  of  research  and 
development,  dissemination  of  information  on  effective  practice,  and  managing  existing 
programs  in  ways  that  capitalize  on  the  benefits  of  educational  technology. 

2 

Engineering 

Capacity 

Building 

There  needs  to  be  a  deliberate  national  reconsideration  of  graduate  education  so  that  the 
open  policy  questions,  the  current  information  gaps,  and  the  contemporary  stresses  are 
systematically  addressed  by  a  suitable  blend  of  university,  industry,  professional  society, 
and  government.  Those  improvements  can  be  made  without  disruption  of  the  traditional 
commitment  to  excellence  in  basic  research  that  has  been,  and  must  continue  to  be,  a 
hallmark  of  the  US  system  of  graduate  education. 

2 

Engineering 

STEM 

Education 

Policy 

The  National  Science  Foundation  should  continue  to  improve  the  coverage,  timeliness, 
and  clarity  of  analysis  of  the  data  on  the  education  and  employment  of  scientists  and 
engineers  in  order  to  support  better  national  decision-making  about  human  resources  in 
science  and  technology. 

In  preparing  this  report,  we  discovered  a  lack  of  the  timely  and  relevant  information  that 
students,  advisers,  and  policy-makers  should  have.  The  National  Science  Foundation 
should  seek  to  improve  timeliness,  increase  detail  on  nonacademic  employment  (which 
now  occupies  most  new  scientists  and  engineers. 

2 

Engineering 

The  burden  of  learning  about  realistic  career  options  should  not  be  left  to  students 
themselves.  We  recommend  the  establishment  of  a  national  database  of  information  on 
employment  options  and  trends.  This  information,  intended  for  use  by  both  students  and 
their  advisers,  should  include,  by  field,  data  on  career  tracks,  graduate  programs 
(including  financial  aid),  time  to  degree,  and  placement  rates.  Departments  should  track 
information  on  their  students— not  only  those  who  go  into  universities  and  4-year 
colleges,  but  those  who  go  into  industry,  government,  junior  and  pre-college  education, 
etc. 

2 

Math 

Design/ 

Methods/ 

Measurement 

Student 
Cognition  & 
Learning 

However,  the  necessary  changes  extend  beyond  the  funding  agencies.  The  research 
community  concerned  with  mathematics  education  must  change  as  well.  Perhaps 
because  mathematics  education  research  has  been  so  poorly  funded  in  the  past,  too  much 
of  the  research  has  taken  place  with  relatively  small  projects,  has  used  diverse  methods 
that  can  make  the  results  difficult  to  compare,  and  has,  therefore,  yielded  too  little 

NSF-14 
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knowledge  that  is  cumulative  and  usable.  The  agenda  that  we  propose  in  this  report  will 
require  greater  collaboration  and  interdisciplinary  action  in  planning,  more  willingness 
on  the  part  of  researchers  to  do  the  work  necessary  to  develop  and  use  common 
measures,  and  more  attention  paid  to  working  collectively  to  build  both  knowledge  and 
practice. 

4 

Technology 

Curriculum  or 

Instructional 

Practic 

Research-based  criteria  for  the  development  of  effective  curriculum  should  also  be 
applied  to  the  development  and  selection  of  educational  courseware. 

5 

Design/ 

Methods/ 

Measurement 

The  conference's  review  of  educational  planning  and  related  activities  suggested  the 
shape  of  a  new,  or  at  least  redefined,  role  for  NERPPB  and  OERI,  and  its  research 
centers,  regional  laboratories,  and  other  assets: 

convening  periodic  meetings  on  educational  research  planning  and  on  such  underlying 
issues  as  standards  of  evidence  and  methodological  progress,  since  there  are  no 
naturally-occurring  forums  for  such  discussions  which  transcend  specific  missions  and 
agendas; 

encouraging  and  coordinating  communications  strategies,  to  place  the  accomplishments, 
promise  and  challenges  of  educational  research  before  its  professional  and  public 
audiences; 

monitoring  the  educational  research  system,  and  building  human  and  institutional 
resources; 

instigating  syntheses  of  all  important  fields  of  educational  research,  to  sum  up  progress 
continually  and  draw  implications  for  policy  and  practice;  and  building  linkages  between 
research  endeavors  and  teachers  in  the  field,  through  consultations,  network  building, 
professional  training  programs,  translation  of  research  findings  into  program  designs  and 
promising  implications  for  the  organization  of  instruction. 

The  agencies  should,  in  other  words,  inhabit  the  space  between  the  research  community, 
the  political  community,  and  the  world  of  practice,  and  help  all  agencies,  associations, 
institutions,  and  individuals  involved  in  educational  research  and  improvement  to  add 
more  value  to  their  own  work  and  to  the  joint  endeavor  of  learning.  The  goal  can  be 
clearly  stated:  in  the  future,  we  must  be  able  to  count  on  educational  progress  that  is 
based  on  ideas  that  have  been  validated  by  well-designed,  well-executed  research,  and 
translated  into  success  by  well-qualified  professionals 

This  has  already 
been  done. 

NSF-8 

6 

Technology 

Initiate  a  major  program  of  experimental  research.  In  view  of  both  the  critical 
importance  of  and  massive  expenditures  associated  with  K-12  education  in  the  United 
States,  the  Panel  recommends  that  an  amount  equal  to  at  least  0.5  percent  of  the  nation's 

This  effort  has  begun 
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aggregate  spending  for  elementary  and  secondary  education  (about  $1.5  billion  at 
current  expenditure  levels)  be  invested  on  an  ongoing  basis  in  federally  sponsored 
research  aimed  at  improving  the  efficacy  and  cost-effectiveness  of  K-12  education. 
Because  no  one  state,  municipality,  or  private  firm  could  hope  to  capture  more  than  a 
small  fraction  of  the  benefits  associated  with  a  significant  advance  in  our  understanding 
of  how  best  to  educate  K-12  students,  this  funding  will  have  to  be  provided  largely  at  the 
federal  level  in  order  to  avoid  a  systematic  underinvestment  (attributable  to  a  classical 
form  of  economic  externality)  relative  to  the  level  that  would  be  optimal  for  the  nation 
as  a  whole. 

6 

Technology 

STEM 

Education 

Policy 

To  ensure  high  standards  of  scientific  excellence,  intellectual  integrity,  and 
independence  from  political  influence,  this  research  program  should  be  planned  and 
overseen  by  a  distinguished  independent  board  of  outside  experts  appointed  by  the 
President,  and  should  encompass  (a)  basic  research  in  various  learning-related 
disciplines  and  on  various  educationally  relevant  technologies;  (b)  early-stage  research 
aimed  at  developing  new  forms  of  educational  software,  content,  and  technology- 
enabled  pedagogy;  and  (c)  rigorous,  well-controlled,  peer-reviewed,  large-scale 
empirical  studies  designed  to  determine  which  educational  approaches  are  in  fact  most 
effective  in  practice.  The  Panel  does  not,  however,  recommend  that  the  deployment  of 
technology  within  America's  schools  be  deferred  pending  the  completion  of  such 
research. 

6 

Technology 

Focus  on  learning  with  technology,  not  about  technology 

Emphasize  content  and  pedagogy,  and  not  just  hardware 

11 

Science 

Math 

The  National  Science  Board  urges  all  stakeholders  in  our  vast  grassroots  system  of  K-12 
education  to  develop  a  nationwide  consensus  for  a  common  core  of  knowledge  and 
competency  in  mathematics  and  science 

11 

Engineering 

Faculty 

Development 

Institutions  must  promote  a  new  balance  and  a  new  linkage  between  teaching  and 
research,  so  that  teaching  is  enlivened  by  investigation  and  research  is  defined  more 
broadly,  and  so  that  faculty  may  be  rewarded  for  educational  scholarship  as  well  as  for 
other  kinds  of  scholarship. 

12 

Design/ 

Methods/ 

Measurement 

Fland-in-hand  with  this  focus  and  strategy  must  come  emphasis  on  more  rigorous 
methods  and  designs,  with  particular  attention  to: 

*  rethinking,  re-imagining  the  possibilities  of  experimental  field  trials  given  new 
technical  tools,  the  complexity  of  the  puzzles  we  seek  to  unravel,  and  the  persuasive 
power  of  randomized  trials  with  policymakers  and  the  public; 

*  designing  processes  ("engineering")  that  systematically  apply  insights  of  research  to 
the  development  of  discrete  education  programs;  and 

97 


*  creating  a  universe  of  reliable  syntheses  of  all-important  areas  of  educational 
research. 

12 

Design/ 

Methods/ 

Measurement 

Another  necessary  element  of  successful  planning  will  be  thorough  going  peer 
participation  and  review,  with  "peer"  denoting  both  the  relevant  community  of  scholars 
(operating  in  study  sections  or  other  continuous  deliberative  bodies)  and  professionals 
from  the  field  (teachers  especially),  participating  fully  in  priority-setting  and  project 
selection,  in  the  design  and  execution  of  collaborative  research  and  in  discussions  about 
the  significance  and  implementation  of  results.  Researchers  and  professionals  must 
develop  a  better  understanding  of  their  mutual  responsibilities  in  performing  research 
and  moving  it  into  practice,  indeed  a  mutual  understanding  about  when  research-based 
knowledge  is  "good  enough"  to  inform  practice  and  policy. 

12 

Design/ 

Methods/ 

Measurement 

The  overriding  sense  of  the  conference  was  that  educational  research  planning  must, 
sooner  rather  than  later,  emphasize  focus  and  selectivity.  Its  inquiries  should  be 
concentrated  on  those  areas  that  the  public  and  profession  believe  are  important  as  well 
as  those  that  will  become  important.  The  touchstone  issue  must  be  student  learning,  with 
a  particular  but  by  no  means  exclusive  emphasis  on  the  challenges  presented  by  ever-as 
growing  diversity  and  inequality.  Selection  of  specific  areas  of  inquiry  must  proceed 
from  assessment  of  what  is  known  and  not  known,  and  of  what  research  opportunities 
are  presented.  Criteria  for  selection  must  be  clear  enough  to  build  strategies  consisting  of 
related  projects  executed  over  time,  and  sometimes  to  exclude  or  redirect  worthy  but  not 
strategically-significant  proposals.  Otherwise,  as  experience  has  shown,  academic  log¬ 
rolling  will  likely  prevail. 

Once  the  problems  of  the  field  are  clearly  specified,  research  plans  should  set  forth  an 
extended  array  of  basic  and  applied  work,  theory  building,  investigations  in  clinical  and 
field  settings,  surveys  and  case  studies  of  field  experiences,  and  syntheses  of  completed 
studies.  The  result  will  be  programs  of  study  that  gain  the  respect  of  the  scientific, 
professional  and  policy  communities,  and  thereby  guarantee  substantial  resources  now 
and  in  the  future. 

14 

Evaluation 

If  parents,  educators,  public  officials,  and  others  who  share  responsibility  for  educational 
outcomes  are  to  discharge  their  responsibility  effectively,  they  should  have  access  to 
information  about  the  nature  and  interpretation  of  tests  and  test  scores.  Such  information 
should  be  made  available  to  the  public  and  should  be  incorporated  into  teacher  education 
and  into  educational  programs  for  principals,  administrators,  public  officials,  and  others. 

16 

Math 

STEM 

Education 

Policy 

To  focus  and  deepen  the  knowledge  base,  an  interagency  Education  Research  Initiative, 
led  by  NSF  and  the  Department  of  Education,  should  be  implemented.  It  should  be 
distinguishable  as  a  joint  venture  within  the  agencies’  respective  research  missions,  and 
cooperatively  funded. 

This  has  already 
been  done 

18 

Miscellaneous 

Support  peer  review  that  facilitates  interdisciplinary  research.  In  reviewing 
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interdisciplinary  research  proposals,  they  should  use  peer  review  groups  that  include 
scientists  in  multiple  disciplines  who  are  themselves  actively  engaged  in 
interdisciplinary  research.  The  system  recently  has  been  modified  at  NIH  with 
encouragement  of  interdisciplinary  and  translational  efforts  in  mind.  Resulting  changes 
should  be  tracked  to  determine  their  impact  on  funding  of  interdisciplinary  grants. 

Continue  and  expand  partnerships  among  funding  agencies  to  provide  the  broadest  base 
for  interdisciplinary  efforts.  These  can  be  inside  an  agency  through  the  formation  of  new 
alliances  among  institutes  or  divisions;  they  can  also  be  among  agencies — such  as  NIH, 
NSF,  the  Department  of  Defense,  and  Department  of  Energy — or  between  the  private 
and  public  sectors. 

Indicate  in  funding  announcements  that  training  is  an  integral  component  of  the 
interdisciplinary  research  project. 

18 

Miscellaneous 

Faculty 

Development 

Support  mid  career  investigators  in  developing  expertise  needed  for  interdisciplinary 
research.  These  programs  should  include  sabbaticals,  career  development  awards,  and 
university-based,  formal  courses  for  faculty  development  to  enhance  interdisciplinary 
and/or  translational  research. 

Continue  funding  for  workshops,  symposia,  and  meetings  to  bring  together  diverse  fields 
to  focus  on  a  particular  scientific  question.  In  such  an  environment,  cross  training  of  the 
investigators  and  encouragement  of  collaboration  would  develop  naturally. 

Support  consortia  and  multi-institutional  programs  that  provide  integration  of  research 
efforts  from  multiple  disciplines. 

18 

Miscellaneous 

Federal  and  private  research  sponsors  should  seek  to  identify  areas  that  can  be  most 
effectively  investigated  with  interdisciplinary  approaches. 

NSF-14 

18 

Miscellaneous 

Funding  agencies  and  universities  should  remove  the  barriers  to  interdisciplinary 
research  and  training  identified  in  this  report.  To  that  end,  funding  agencies  should: 

Require  commitments  from  university  administration  to  qualify  for  funding  for 
Interdisciplinary  efforts. 

Facilitate  interactions  among  investigators  in  different  disciplines  by  funding  shared 
and  core  facilities. 

Encourage  legislation  to  expand  loan  repayment  programs  to  include  investigators 
outside  NIH  who  are  engaged  in  funded  interdisciplinary  and  translational  research. 
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Support  peer  review  that  facilitates  interdisciplinary  research. 

Continue  and  expand  partnerships  among  funding  agencies  to  provide  the  broadest  base 
for  interdisciplinary  efforts. 

Indicate  in  funding  announcements  that  training  is  an  integral  component  of  the 
interdisciplinary  research  project. 

19 

Miscellaneous 

Establishing  a  national  database  for  improving  teaching  of  science,  mathematics  and 
technology.  Nearly  every  state  is  at  some  stage  of  developing  databases  and  other 
resources  for  its  teachers  to  enable  them  to  understand  and  teach  to  state  standards  in 
science  and  mathematics.  While  ever  state's  standards  differ  to  some  degree,  most  of 
them  are  based  at  least  in  part  on  the  national  standards  for  science  and  mathematics. 

Thus,  it  is  likely  that  great  deal  of  overlapping  effort  is  taking  place.  If  the  federal 
government  could  establish  a  national  database  for  improving  the  teaching  of  science, 
mathematics,  and  technology  that  would  allow  teachers  to  easily  access  information 
from  their  state  and  elsewhere,  teaching  of  these  disciplines  could  vastly  be  vastly 
improved.  The  National  Science  Foundation's  National  digital  Library  project  could 
serve  as  the  focal  point  for  such  a  compendium  of  information.  The  CSTMP 
recommends  that  future  Requests  for  Proposals  include  specific  requests  to  develop  this 
national  database  and  library  on  teaching  of  science,  mathematics,  and  technology. 

19 

Miscellaneous 

Teacher 

Development 

Universities  whose  primary  mission  includes  education  research  should  set  as  a  priority 
the  development  and  execution  of  peer-reviewed  research  studies  that  focus  on  ways  to 
improve  teacher  education,  the  art  of  teaching,  and  learning  for  people  of  all  ages.  New 
research  that  focuses  broadly  on  synthesizing  data  across  studies  and  linking  it  to  school 
practice  in  a  wide  variety  of  school  settings  would  be  especially  helpful  to  the 
improvement  of  teacher  education  and  professional  development  for  both  prospective 
and  experienced  teachers.  The  results  of  this  research  should  be  collated  and 
disseminated  through  a  national  electronic  database  or  library. 

19 

Miscellaneous 

STEM 

Education 

Policy 

Establishing  a  national  database  for  improving  teaching  of  science,  mathematics  and 
technology.  Nearly  every  state  is  at  some  stage  of  developing  databases  and  other 
resources  for  its  teachers  to  enable  them  to  understand  and  teach  to  state  standards  in 
science  and  mathematics.  While  ever  state's  standards  differ  to  some  degree,  most  of 
them  are  based  at  least  in  part  on  the  national  standards  for  science  and  mathematics. 

Thus,  it  is  likely  that  great  deal  of  overlapping  effort  is  taking  place.  If  the  federal 
government  could  establish  a  national  database  for  improving  the  teaching  of  science, 
mathematics,  and  technology  that  would  allow  teachers  to  easily  access  information 
from  their  state  and  elsewhere,  teaching  of  these  disciplines  could  vastly  be  vastly 
improved  .  The  National  Science  Foundation's  National  digital  Library  project  could 
serve  as  the  focal  point  for  such  a  compendium  of  information.  The  CSTMP 
recommends  that  future  Requests  for  Proposals  include  specific  requests  to  develop  this 
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national  database  and  library  on  teaching  of  science,  mathematics,  and  technology. 

19 

Science 

Curriculum  or 

Instructional 

Practice 

Student  achievement  with  a  model  of  cognition  and  learning  that  can  serve  as  the  basis 
for  assessment  design  for  all  areas  of  the  school  curriculum. 

19 

Miscellaneous 

Teacher 

Development 

Research  that  focuses  broadly  on  synthesizing  data  across  studies  and  linking  it  to  school 
practice  in  a  wide  variety  of  school  settings  would  be  helpful  to  the  improvement  of 
teacher  education  and  professional  development  for  prospective  and  experienced 
teachers. 

20 

Engineering 

Capacity 

Building 

An  important  first  step  is  for  institutions  to  take  a  census  of  their  postdoctoral 
populations.  Many  institutions,  especially  universities,  have  no  accurate  count  or 
counting  mechanism. 

Reporting  and  tracking:  Funding  organizations  should  track  postdocs  after  they  leave 
individual  labs,  to  help  determine  whether  that  lab  should  continue  to  receive  funding  for 
postdoc  training;  funders'  already  use  the  yardstick  of  post-appointment  performance  for 
training  grants.  Tracking  might  be  done  via  a  web  site,  and  should  provide  useful 
information,  such  as  numbers,  characteristics,  and  subsequent  employment.  When  a 
postdoc  experience  ends,  the  organization  can  use  a  “virtual  exit  interview”  (or  some 
other  mechanism)  to  determine  the  quality  of  a  postdoc  experience  and  to  identify 
problems.  Such  reviews  of  outcomes  may  help  federal  organizations  comply  with  the 
Government  Performance  and  Results  Act. 

22 

Miscellaneous 

Curriculum  or 

Instructional 

Practice 

From  this  perspective,  one  can  envision  the  need  for  a  comprehensive  program  of  use- 
driven  strategic  research  and  development  focused  on  issues  of  improving  classroom 
learning  and  teaching.  The  facts  that  schools  and  classrooms  are  the  focus  and  that 
enhanced  practice  and  learning  are  the  desired  goals  render  the  program  of  research  no 
less  important  with  respect  to  advancing  the  theoretical  base  for  how  people  learn. 

Indeed,  many  of  the  advances  described  in  this  volume  are  the  product  of  use-inspired 
research  and  development  focused  on  solving  problems  of  classroom  practice. 

22 

Design/ 

Methods/ 

Measurement 

It  is  worth  noting  that  a  wide  array  of  quantitative  and  qualitative  methods  drawn  from 
the  behavioral  and  social  sciences  are  employed  in  education  research.  The  methods 
often  vary  with  the  nature  of  the  learning  and  teaching  problem  studied  and  the  level  of 
detail  at  which  issues  are  pursued.  Given  the  complexity  of  educational  issues  in  real- 
world  contexts  in  which  variables  are  often  difficult  to  control,  the  types  of  “use- 
inspired”  research  envisioned  here  will  necessarily  demand  a  variety  of  methods.  These 
will  range  from  controlled  designs  to  case  studies,  with  analytic  methods  for  deriving 
conclusions  and  inferences  including  both  quantitative  and  qualitative  procedures  of 
substantial  rigor.  To  build  an  effective  bridge  between  research  and  practice,  such  a 
multiplicity  of  methods  is  not  only  reasonable,  it  is  essential.  No  single  research  method 
can  suffice. 
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22 

Miscellaneous 

Student 
Cognition  & 
Learning 

Bridge  theory  and  practice  by  conducting  “use-inspired”  research  focused  on  improving 
classroom  learning  and  teaching. 

32 

Design/ 

Methods/ 

Measurement 

Accumulated  knowledge  and  ongoing  advances  from  the  merger  of  the  cognitive  and 
measurement  sciences  should  be  synthesized  and  made  available  in  usable  forms  to 
multiple  educational  constituencies.  These  constituencies  include  educational 
researchers,  test  developers,  curriculum  specialists,  teachers,  and  policy  makers. 

32 

Design/ 

Methods/ 

Measurement 

STEM 

Education 

Policy 

Federal  agencies  and  private-sector  organizations  concerned  with  issues  of  assessment 
should  support  the  establishment  of  multidisciplinary  discourse  communities.  The 
purpose  of  such  discourse  would  be  to  facilitate  cross-fertilization  of  ideas  among 
researchers  and  assessment  developers  working  at  the  intersection  of  cognitive  theory 
and  educational  measurement. 

NSF-14 

32 

Design/ 

Methods/ 

Measurement 

Student 
Cognition  & 
Learning 

Research  on  new  statistical  measurement  models  and  their  applicability  should  be  tied  to 
modern  theories  of  cognition  and  learning. 

34 

Evaluation 

Evaluation  & 
Assessment 

The  committee’s  criteria  forjudging  test  quality  include  the  following:  tests  should  have 
a  statement  of  purpose;  systematic  processes  should  be  used  in  deciding  what  to  test  and 
in  assuring  balanced  and  adequate  coverage  of  these  competencies;  test  materials  should 
be  tried  out  and  analyzed  before  operational  decisions  are  made;  test  administration  and 
scoring  should  be  uniform  and  fair;  test  materials  and  results  should  be  protected  from 
corruptibility;  standard-setting  procedures  should  be  systematic  and  well  documented; 
test  results  should  be  consistent  across  test  forms  and  scorers;  information  about  tests 
and  scoring  should  be  available  to  candidates;  technical  documentation  should  be 
accessible  for  public  and  professional  review;  validity  evidence  should  be  gathered  and 
presented;  costs  and  feasibility  should  be  considered  in  test  development  and  selection; 
and  the  long-term  consequences  of  licensing  tests  should  be  monitored  and  examined. 

34 

Evaluation 

Teacher 

Development 

To  fairly  and  accurately  judge  the  quality  of  teacher  education  programs,  federal  and 
state  officials  need  data  on  a  wide  variety  of  program  characteristics  from  multiple 
sources.  Other  indicators  of  program  quality  might  include  assessment  data  for  students 
in  relation  to  course  and  program  benchmarks,  employer  evaluations,  and  district  or  state 
evaluations  of  beginning  teaching.  Other  indicators  might  include  information  on  course 
requirements  and  course  quality,  measures  of  the  amount  and  quality  of  field 
experiences,  and  evidence  of  opportunities  to  work  with  students  with  special  learning 
needs  and  students  with  diverse  backgrounds.  Data  on  the  qualifications  of  program 
faculty,  the  allocation  of  resources,  and  the  adequacy  of  facilities  might  be  considered. 

The  qualifications  of  students  at  entry  to  teacher  education  programs  also  should  be 
included 

42 

Miscellaneous 

STEM 

Education 

The  National  Science  Foundation  and  the  Department  of  Education  must  spearhead  the 
Federal  contribution  to  SMET  education  research  and  evaluation. 

This  has  already 
been  done 
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Policy 

42 

Science 

Math 

Schoolwide  & 

Systemwide 

Reforms 

Overall,  the  investment  should  increase — by  the  Federal  government,  private 
foundations,  and  other  sponsors — in  research  on  schooling,  educational  systems  more 
generally,  and  teaching  and  learning  of  mathematics  and  science  in  particular. 

48 

Engineering 

STEM 

Education 

Policy 

A  multidisciplinary,  precursor  committee  drawn  from  stakeholders  in  STEM  education 
should  oversee  the  establishment  of  a  comprehensive  national  strategy  to  achieve  IT- 
transformed  STEM  education. 

48 

Technology 

STEM 

Education 

Policy 

A  multidisciplinary,  precursor  committee  drawn  from  stakeholders  in  STEM  education 
should  oversee  the  establishment  of  a  comprehensive  national  strategy  to  achieve  IT- 
transformed  STEM  education. 

50 

Engineering 

Curriculum  or 

Instructional 

Practice 

The  Commission  provides  a  thorough  assessment  of  the  state  of  mathematics  and 
science  teaching  and  a  comprehensive  set  of  measures  to  improve  it.  Strengthening 
instruction  is  critical  to  getting  more  students  at  the  top  level  of  achievement  needed  to 
enter  science  and  engineering,  and  critical  to  reducing  the  wide  disparity  by  race  and 
ethnicity  that  now  exists  at  the  Proficient  level  of  performance  on  the  NAEP  math  and 
science  assessments. 

50 

Engineering 

Sources  of  Inequality  and  Mediocrity.  It  is  probably  impossible  to  use  the  tools  of  social 
science  research  to  unravel  all  the  determinants  of  educational  achievement,  although 
sufficient  brainpower,  unlimited  access  to  all  the  relevant  data,  and  adequate  resources 
for  analyzing  it,  might  get  us  reasonably  close.  The  undertaking  seems  as  monumental  as 
the  project  to  map  the  human  genome.  Even  if  we  succeeded,  in  the  end  we  would  be 
left  with  a  voluminous  list  of  factors  and  conditions  found  to  be  "associated"  (correlated) 
with  academic  achievement;  actual  "causes"  would  remain  elusive. 

51 

Evaluation 

Evaluation  & 
Assessment 

Create  the  test  assessments  that  are  aligned  with  21st  century  skills. 

52 

Evaluation 

Evaluation  & 
Assessment 

They  must  bring  outstanding  individuals  into  the  planning  of  the  work  and  into  the 
selection  of  the  proposals,  people,  and  institutions  that  can  carry  it  out  most  effectively. 
They  must  arrange  for  the  regular  critical  review  and  evaluation  of  what  has  been 
supported  and  what  has  been  learned,  and  they  must  make  adjustments  in  the  program 
that  are  suggested  by  such  review. 

52 

Math 

While  some  issues  surrounding  mathematics  education,  particularly  concerning  what  it 
is  that  students  should  know  and  be  able  to  do  in  mathematics,  involve  inherently 
political  decisions,  we  believe  that  most  of  these  issues  can  be  illuminated  by 
appropriate  and  timely  research  and  evaluation.  Current  debates  surrounding 
mathematics  education  have  not  been  adequately  informed  by  the  work  of  the  research 
community.  Because  of  this,  these  debates  have  often  been  undisciplined  and  overly 
contentious.  The  program  of  research  and  development  envisioned  in  this  report  is 
intended  to  move  the  nation  beyond  these  debates  to  significant  improvements  in  student 
learning.  Achieving  what  we  envision  will  require  building  and  enhancing  a  vigorous 
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and  critical  research,  development,  and  practice  community.  Within  such  a  community, 
we  hope  that  debate  among  those  with  varying  and  competing  views  concerning 
standards  of  proficiency,  curricular  designs,  pedagogical  styles,  and  assessment  methods 
will  evolve  into  a  discourse  that  is  based  less  on  ideology  and  more  on  evidence. 

The  RAND  Mathematics  Study  Panel  asserts  that  our  nation’s  future  well-being  depends 
on  shifts  in  how  research  and  development  in  mathematics  education  are  designed, 
supported,  coordinated,  and  managed.  Mathematical  proficiency  is  one  of  the  most 
important  capabilities  needed  by  the  people  of  the  United  States  in  the  21st  century. 
Achieving  mathematical  proficiency  equitably  will  require  the  targeted  investment 
recommended  in  this  report. 

52 

Math 

STEM 

Education 

Policy 

The  program  we  propose  will  require  contributions  of  individuals  with  wide  ranging 
skills  and  sustained  commitment  on  the  part  of  the  federal  offices  that  support  research 
and  development  in  mathematics  education.  The  staff  in  these  offices  must  be  adept  at 
engaging  the  research  and  education  communities  in  the  partnership  that  we  have  argued 
is  necessary  to  move  forward  with  the  program  we  propose.  Federal  office  staff  must 
organize  the  program  in  ways  that  ensure  the  rigor,  cumulativeness,  and  usability  of  the 
research  and  development. 

52 

Math 

STEM 

Education 

Policy 

The  RAND  panel  has  also  made  proposals  on  how  the  research  and  development 
program  should  be  conducted.  New  approaches  to  program  funding  and  new 
management  styles  are  recommended.  These  approaches  should  ensure  that  the 
supported  work  incorporates  effective  scientific  practices,  uses  methods  appropriate  to 
the  goals  of  the  component  projects,  and  that  the  program  builds  knowledge  over  time. 
Further,  interventions  should  be  rigorously  tested  and  revised  through  cycles  of  design 
and  trial. 

52 

Math 

Moreover,  both  funders  and  researchers  must  develop  better  ways  to  engage  the 
practitioner  community  in  this  work.  It  is  not  enough  to  have  a  single  practitioner 
serving  as  a  member  of  a  peer  review  group  or  serving  on  a  study  panel.  Research  and 
development  initiatives  must  be  more  solidly  informed  and  guided  by  the  wisdom  of 
practice.  New  institutions  that  can  engage  researchers  and  practitioners  in  joint  work  are 
needed.  New  partnerships  between  research  institutions  and  schools  and  school  districts 
must  be  forged.  The  research  and  development  program  that  the  RAND  panel  proposes 
is  unlikely  to  produce  usable  results  if  progress  is  not  made  in  bridging  research  and 
practice 

54 

Miscellaneous 

Identify  key  research  investment  opportunities  regarding  children  and  adolescents  —  to 
achieve  the  overarching  goals  outlined  in  the  strategic  plan.  These  investment 
opportunities  should  highlight  the  need  for  and  the  benefits  of  a  multi-agency, 
coordinated  approach  to  scientific  research  concerning  young  people.  The  Office  of 
Science  and  Technology  Policy  (OSTP)  and  the  Office  of  Management  and  Budget 
(OMB)  would  play  an  integral  role  in  working  with  the  IWG  to  develop  a  coordinated 
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strategy,  including  budgetary  issues,  in  which  agencies  can  collaborate  on  research 
problems  of  national  concern.  These  problems  include  strengthening  the  collection  of 
reliable  indicators  of  child  and  adolescent  well-being  and  the  research  that  provides  us 
with  knowledge  of  the  factors  that  influence  these  indicators  over  time.  A  coordinated 
research  strategy  on  optimal  human  development  from  early  childhood  into  young 
adulthood,  particularly  on  factors  supporting  learning,  should  be  a  high  priority. 

Identify  mechanisms  to  strengthen  research-policy  linkages  --  not  only  among  NSTC, 
DPC,  Federal  agencies  and  State  and  local  government,  but  also  among  relevant  non¬ 
governmental  organizations  and  other  public  and  private  sector  parties  at  the  national, 
regional,  and  community  levels.  Such  extensive  linkages  are  needed  to  ensure  that 
research  knowledge  generated  by  Federal  agencies  and  other  entities  are  effectively  used 
to  inform  policy  and  program  development  regarding  children,  youth,  and  families. 

54 

Miscellaneous 

There  have  been  many  programs  of  educational  research  supported  by  federal  agencies 
and  private  foundations  designed  to  improve  student  outcomes.  They  have  generated 
important  research-based  knowledge,  but  their  efforts  have  not  effectively  penetrated 
educational  practice.  Education  does  not  presently  function  like  medicine  or  agriculture, 
where  close  linkages  between  research  and  practice  have  had  major  influences  on  both. 

Three  sets  of  powerful  but  underutilized  resources  convince  us  that  a  SERP  research  and 
development  enterprise  could  support  genuine  improvements  in  student  achievement:  (1) 
advances  in  the  disciplines  with  relevance  to  education  )  cognitive  science, 
developmental  psychology,  organizational  theory)  that  are  largely  untapped;  (2)  natural 
variations  in  educational  practice  that  have  not  been  studies  systematically;  and  (3) 
innovations  in  educational  practice  and  policy  that  have  been  demonstrated  to  be 
effective,  at  least  in  particular  settings,  but  have  not  been  sufficiently  developed  or 
studies  for  purposes  of  moving  to  scale  so  that  they  have  broad  influence  on  student 
outcomes.  The  problem  of  effectively  capitalizing  on  these  resources  poses  several 
challenges: 

There  is  currently  no  institution  in  which  education  practitioners  and  researchers  from  a 
variety  of  disciplines  are  provided  with  support  to  interact,  collaborate,  and  leam  from 
each  other.  Thus,  researchers  often  fail  to  bring  important  understandings  to  the  stage  of 
usability,  and  practitioners  have  no  way  either  to  analyze  and  systematize  their  own 
wisdom  of  practice  or  to  influence  the  directions  and  shape  of  the  research  agenda. 
Moreover,  researchers  have  little  opportunity  to  see  and  try  to  understand  the  variety  of 
practices  and  outcomes  that  characterize  the  operational  setting. 

There  is  no  site  where  a  carefully  vetted  knowledge  base  about  instructional  innovation, 
school  reform,  and  education  policy  resides  and  accumulates. 
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There  are  few  vehicles  for  conceptually  coherent  research  planning  so  that  research 
agendas  tend  to  resemble  topical  lists  responsive  to  neither  the  strengths  of  research  nor 
the  complexities  of  practice. 

56 

Math 

STEM 

Education 

Policy 

To  this  end,  BEST  recommends: 

•  Federal  agencies  should  adopt  and  enforce  criteria  taking  diversity  into  account  in 
awarding  education  and  research  grants  to  institutions  of  higher  education. 

58 

Math 

Evaluation  & 
Assessment 

A  curricular  program's  effectiveness  should  be  ascertained  through  the  use  of  multiple 
methods  of  evaluation,  each  of  which  should  be  a  scientifically  valid  study.  Periodic 
synthesis  of  the  results  across  evaluations  studies  should  also  be  conducted. 

58 

Math 

Field  Initiated 
Topics 

Six  qualities  of  scientific  research  were  identified  as  cmcia|: 

1.  Posing  significant  questions  that  can  be  investigated  empirically; 

2.  Linking  research  to  relevant  theory; 

3.  Using  methods  that  permit  direct  investigation  of  the  question; 

4.  Providing  a  coherent  and  explicit  chain  of  reasoning; 

5.  Replicating  and  generalizing  across  studies;  and 

6.  Disclosing  research  to  encourage  professional  scrutiny  and  critique. 

62 

Science 

Equity- 

Race/Ethnicity 

Direct  research  funds  toward  increasing  diversity  in  the  field. 

